Are you comparing the results of one question between two groups, or comparing the sum of several questions between two groups?

Thanks all for your response. Actually i have first question like this

Q1. Which software development method do you use?

1- agile method 2- heavyweight methods

for example i get 50 responses , 30 people say they use agile and 20 say they use heavyweight methods. now i have 6 more questions to analyze the satisfaction of users on different aspects.i use charts to show percentages for both methods seperatly. following are the questions.

Q1.How satisfied you are with your software development method?

Q2.To what extent, do you agree that your software development method helps you to have understandable communication with your customers? Q3.To what extent, do you agree that your software development method help you to fulfill your customer’s requirements / demands? Q4.To what extent, do you agree that your software development method helps you to develop high quality software? Q5.Towhat extent, do you agree that your software development method helps you in risk analysis? Q6.To what extent, you agree that your software development method support you in rapid development?

i want to use other technique to compare both methods seperatly. attached is the paper i used for my thesis.

thanks hassan for your reply. do you mean that for example in following question

Q2.To what extent, do you agree that your software development method helps you to have understandable communication with your customers?

i should take method as independent variable and understandable communication satisfaction as dependent variable. then i should calculate correlation coefficient by spss for both methods seperataley on the basis of first question?

Q1. Which software development method do you use?

1- agile method 2- heavyweight methods

then compare results of both methods. am i right?

It sounds like your main goal is to use the answer from Q1 to treat the respondents as two groups. That is, use Q1 as your independent variable and each of the following questions (2 - 6) individually as the dependent variables.

To do this, you probably want to conduct a statistical test to get a p-value, and also look at the effect size.

You should probably treat your Likert results as ordinal variables. Appropriate methods to compare ordinal variables for two groups are ordinal regression, and Cochran-Armitage test. Mann-Whitney may also be appropriate, though some disagree. M-W does have the advantage of being easy to conduct with most statistical software and probably being familiar to your audience.

For effect size, Freeman's theta and epsilon-squared are appropriate.

Alternatively, you can code your groups as a numeric and use Kendall correlation. In this case kendall's tau can be used as a measure of effect size. Don't use tau-a for this situation.

Thanks Salvatore. What do you mean by effect size? actually i have total 410 responses where 22 people use Heavyweight methods and rest use Agile methods. There is huge difference , can i compare two groups on this difference? Spearman's rank correlation can be used ? dependent variable is communicational satisfaction and independent variable are two methods, suppose i calculate spearman co efficient separately for both methods and i get value more near to 1 for agile than heavyweight then agile will be turned out to be better for communicative satisfaction for users as compared to heavyweight? and what about t - test ? please clear it. thanks

1) The tests I mentioned shouldn't be particularly disturbed by unequal sample sizes. However, with very different sample sizes, your test may lack power. (See: https://stats.stackexchange.com/questions/40342/mann-whitney-u-test-with-unequal-sample-sizes )

2) I think I recommend Kendall correlation over Spearman correlation for Likert data. As the following paper says, "We preferred Kendall's τ-b over Spearman's rank correlation coefficient because it is less sensitive to tied ranks and outliers." But I think Spearman could be used as well. (Article Pitfalls in the statistical examination and interpretation o...

3) I don't know what "calculate spearman co efficient separately for both methods" means.

4) Effect sizes are a measure of e.g. the difference between two groups, that is not dependent on sample size. They are analogous to the r-squared value for linear regression. It is important to always include some measure of effect size, or another way to describe the size of the effect. Reporting only a p-value gives the reader a very incomplete understanding of the results. For a telling example, the p-value for the correlation for the data in the attached plot is p = 0.01, suggesting a significant correlation. However, Pearson's r is 0.1, which is rather low. It is necessary in this case to either report r, or to show a plot of the data to give the reader a more complete picture of the extent of the correlation. In your case, you might present either an appropriate effect size (e.g. Freeman's theta, epsilon-squared, Kendall's tau-b), show two bar plots similar to histograms for the frequency of responses for each option in the Likert item ( http://rcompanion.org/handbook/images/image061.png ), or minimally present the medians for each group.

Thanks Salvatore S. Mangiafico . I have to check normal distribution of data combine or in separate group for each dependent variable to perform M.W test?

Mann-Whitney does not assume anything about the distribution of the data.

Please look at sheet and suggest can i perform mann whiteny test on it? Does it fullfill all assumptions to perform MW?

Appropriate data for the Mann-Whitney test are:

1) Dependent variable is ordinal, interval, or ratio.

2) Data are arranged in two groups. Or, Independent variable is a factor, nominal, or grouping variable, with two levels.

3) Observations are independent. That is, they are not paired between the groups, or autocorrelated.

The null hypothesis is: 1) If the distribution of the two groups are similar in shape and spread, "The median of the two groups is equal". 2) If the distribution of the two groups are not the same in shape and spread, "The values for the two groups are stochastically equal."

I have very basic knowledge of research methods. Please can you check the attached sheet and verify the last normal distribution clause. It will take only a minute. I shall be very thankful to you.

What sheet?

This attached document.

There are q-q plots on pages 17, 23, 35, 47, that all appear to be the same. Is this what you are asking about? The data appear rather skewed. This link is helpful for interpreting q-q plots. https://stats.stackexchange.com/questions/101274/how-to-interpret-a-qq-plot

Can i perform mann whiteny test on this data ? What is your thought after watching these plots and figures?

Mann-Whitney makes no assumptions about the distribution of the data, so there is nothing in those plots that would make me think that can't use the test. The assumptions of the test are listed above.

Can we perform m.w two groups that differ hugely. Like in my case 380 and 22?

My understanding is that the test is valid, but that the power of the test decreases as the sample sizes become more different. The power is the ability to detect a difference if one in fact exists.