I have thought about this question a lot, and my concern is that there are a group of subjects that aren't preferable to be investigated by interviews. Because the results would be misleading. Very misleading! Let me explain. Imagine you want to know to what extent certain experiences has occurred in a certain group of the society. For example how many people experienced headaches in the last month. Then one can conduct a set of interviews and ask relevant questions to the respondents about whether they had headaches, to what extent they had, what time in a daytime, and so on. In this case interviews seems to be a valid method because the respondents can be quite precise in their experiences and can report valid data (if all due methodical precautions are respected).

Now, my concern is that in software engineering most of the subjects we are investigating are of much complex types and respondents' opinions in most cases has nothing to do with the reality. For example, we conduct interviews to investigate the reasons of why the tractability of software requirements cannot be established properly in industry. Say we select 100 respondents from 4 companies and ask them relevant questions on this subject. Then we compile the results with inductive methods. So far the kind of results that we get in this kind of studies are either simplistic or wrong. In the former case we get answers like, because the requirements are changing or because the practitioners don't have time. That requirements are changing was found in 1990s or that practitioners don't have time is kind of tautological isn't it? This kind of answers don't seem to provide any value. In the latter case we get answers like because managers don't prioritize tractability tasks. Not really true, tractability tasks are part of system development task and the reasons for not prioritizing is more profound: such as the entire coding world and requirements world are disconnected in the development each of the practitioners concerned with own problems. As a result code and requirements management systems aren't integrated. And if we go one step deeper - why aren't they integrated? then we find that there is not enough believe among the software engineers that code and requirements should be treated as one system and one system development. So there is a culture and maturity problem too. But this means that the questions we investigated are a part of much complex problem. And how on earth do we expect practitioners giving wise answers to this complex problems?

Shouldn't we be more considerate on where we use interviews? And how?

Similar questions and discussions