Tamer, you and Ian are having a having a good dialog about this and it's not something that I can solve, but let me add a little bit from decades of survey work. It's often best to devote your resources to a carefully thought-out sample of something you can define and sample from and get a good response rate that to take some kind of shot gun approach and just accept everything that comes in. By my thinking most web surveys aren't really surveys at all, because they are just based on a limited number of people just clicking based on their willingness to click on the survey and fill it out. The authors of the so-called survey don't even know how many unique individuals saw the invitation nonetheless who they might be and so on. Way before the Internet, such surveys were called SLOPs, self-selected listener opinion polls and were considered meaningless. So whatever you do, you are probably best off working for meaningful lists, such as Agile developers, assuming you could get such a list or permission to mail to the list, or user groups with finite membership, than desperately posting invitations having no idea how many you may be reaching. It's harder to know how to target users, but it might be sensible to work with a limited number of sites that can identify unique users in a given period of time and sort them into types to represent a diversity of types, and you might have some kind of basis to figure a response rate. Just some ideas off the top of my head. Bob

A tricky one to answer Tamer - particularly in today's global society. Let's say that in a given country most of the population has access to the Internet. You would need a sample almost the size of those chosen for a national census I.e. most of the population - or at least a representative proportion. I.e. 10%. Of course it depends on the question, methodology etc - but, without a specific question about a specific population, Internet usage generally incorporates such a wide and varied population that your sample would have to include many different demographics and variables.

Thanks a lot. Let's take Malaysia as an example. Number of internet users 19.2 million. The sample size: 385, is it correct?

No! For a 100% response rate, a random sample size of 384 is required, as long as you are asking a YES/NO question and are happy with a sample error of 5%.

Excuse me guys - how did you come up with a number like 384/385? Seems quite exact for a total population of 19.2 million. I would hardly say that this sample size could provide generalisable results - even with power statistics. As I previously said Tamar, impossible to know without knowing what you are attempting to do and how.

I too, am perplexed by the 384 vs. 385 business. I can't imagine what you are using to come up with that number and worrying about that difference. I can say, it isn't going to matter much whether it's 1.9 million, 19 million or 192 million. I agree with Dean's 2nd answer, it's going to matter what your questions are and what precision you want in your results. If you are going to do the study on the Internet, the problem is going to be getting a scientific sample at all. Bob

If you wanted to generalize the results to the whole population of earth you wouldn't be thinking of your sample as a percentage of the human population. Similarly 19.2 million is high enough to not matter: i.e. as long as your sample is random you should not estimate the sample size as a percentage of the population

As others explained, it really depends on your research questions and hypotheses.

Thanks guys. My main research question is:

What are the preferences (quality attributes) from users' perceptions while using websites?

Ok. So the magic is that if you ask binary questions, like "Are you happy with the speed of your Internet connection? YES/NO"; "Do you watch videos via the Internet? YES/NO" AND your report reader is happy with a 5% sample error in each answer, and you are confident of getting a 100% response rate, then the statistical formula requires you to have only 384 respondents, chosen really at random.

Sure, if you want a 0.0000% sample error, then you have to have a 100% response rate from 19.2 million responding Internet users. But who needs such certainty and which poor researcher can afford that? And you thought that statisticians were worthless people and could not save you money!

(The only time that the report reader might be a bit unhappy with a 5% sample error is if the answer is a close tie. For example, if 50% respond YES, then it might be 52.5% of the population of users who believe YES OR maybe 47.5% of the users believing YES, we cannot be sure.)

Actually, my research topic is on users' and developers' perceptions of quality in websites developed using agile software development approach. My study is purely descriptive, I do not have variables and hypotheses.

The survey questions are the same for users and developers. The population of developers is small, so I found one reference says that I should take all the population since it is too small. While for users, I do not know!!

I do not have much experience in sampling and analysis and I just need to know which sample and analysis I should use.

It is too dangerous to survey a sample of developers in your country. Try to survey them all. Getting a list of their E-mail addresses is problematic.

Not all Web sites have been developed using agile technologies. So the population of users is NOT 19.2 Million! You have now made yourself an impossible task of identifying (1) Which of all the Web sites in the WORLD were developed using agile technologies. They do not have badges saying "MADE IN AGILELAND"! (2) Which of your country's Internet users actually visit these sites (and can recognise that they were built using agile technologies?!!) You cannot get a list of their E-mail addresses!!!

I do not understand how you can describe experiences without measurements, which implies variables and hypotheses. I think you need to do some hard thinking.

In fact, I am facing some difficulties because of :

1. The focus of my study is on two sides: users and developers.

2. Since I am studying quality attributes from users' and developers' perspectives, I do not want to be too specific or general. That's why I chose the context of web applications. The context has been chosen after conducting a preliminary study with a group of agile developers. The results of the study indicate that usually agile is used to develop web applications. Moreover, the results show that the quality attributes are unclear and need to be defined from users' and developers' perspectives. Finally, results recommend to investigate quality attributes from both perspectives and to develop a process or model to match both perspectives.

We make a note here that the study was conducted to support our review for the literature.

I would be very grateful Ian if you could please give some advice and suggestions.

OK. I am on the same page as you now. The problem is not really sample size. As I see it, your problem is how to gain access to data.

I suggest that you gather all quality attributes that you find in the literature, add any missing ideas that you have. These attributes can inform your questionnaire. Yes, it seems natural to limit your study to the development / use of Web applications, because this is where the effort is. I agree that quality attributes are nebulous. This itself is an open field for research. The attributes as seen by developer and user are likely to be different, and this also is a good area to extend your research. There should be a consensus, and probably a difference. The difference is what will make your results interesting. This could have practical benefits in allowing users' and developers' perspectives to become aligned. This would be nice, but....

Can you get access to a larger set of agile software Web developer's? Can you find what sites they have developed?? Can you find our who the users of those sites are??? Will you be able to contact them????

For developers, yes can while for users can not. We just take a random sample of users and ask them generally about quality attributes of their preferred website.

But what if that preferred Website was one of the ninety-nine % that was NOT developed using agile technology? (100*384=38400 users!)

So, what?

38400 user responses do not easily fall into your laptop to give you 384 usable responses.

I am confused !! and do not know what to do !!

Tamer, you and Ian are having a having a good dialog about this and it's not something that I can solve, but let me add a little bit from decades of survey work. It's often best to devote your resources to a carefully thought-out sample of something you can define and sample from and get a good response rate that to take some kind of shot gun approach and just accept everything that comes in. By my thinking most web surveys aren't really surveys at all, because they are just based on a limited number of people just clicking based on their willingness to click on the survey and fill it out. The authors of the so-called survey don't even know how many unique individuals saw the invitation nonetheless who they might be and so on. Way before the Internet, such surveys were called SLOPs, self-selected listener opinion polls and were considered meaningless. So whatever you do, you are probably best off working for meaningful lists, such as Agile developers, assuming you could get such a list or permission to mail to the list, or user groups with finite membership, than desperately posting invitations having no idea how many you may be reaching. It's harder to know how to target users, but it might be sensible to work with a limited number of sites that can identify unique users in a given period of time and sort them into types to represent a diversity of types, and you might have some kind of basis to figure a response rate. Just some ideas off the top of my head. Bob

Robert writes wise words. Listen to them.

Indeed, Bob's and Ian's comments are very valuable. Let me add this, in case you were able to determine your "finite" population, you can use this formula for sample size determination: n= N/1+N(square of e). This assumes a very large sample and a confidence level of 95.44 percent. For the selection, you can use random sampling. The Sampling function in the Excel Analysis Tool pak can facilitate th\s.

In case I am not able to determine my "finite" population?

For this kind of participants (people), it would not be advisable to use the formula for unknown or infinite population because you cannot make a sampling frame from where to randomly select your sample. Your other option is to resort to non-probability sampling (convenience, quota, purposive, judgmental, snowball, etc.). But take note that in this method, you cannot make an inferential analysis.

Thank you very much Eddie