Currently i'm a PhD student and with my team we are developing a middleware solution for IoT applications equipped with constrained devices.
to demonstrate this proposal and as we found that there are two type of evaluation parameters: qualitative and quantitative, the first generally by a textual comparison with other developed solutions in which the goals are achieved; but the second one, unfortunately we are confused with it due to this type is depended on the target of the application (as i understand ), in our case the proposed solution is still a prototype and need advanced development to test it in real application.
my question is how and what are the parameters should be evaluated to demonstrate our idea without comparing it with other solution that i consider them so heavy and complicated.
Hi Ayoub
Qualitative research, usually collected by interviews, focus groups or observation seek to collect a rich depth of data from such studies. Quantitative data, as the name suggest seeks to collect vast quantities of data, based on specific measurements of numerical data which can be used by statistical analysis methods to determine the success, or otherwise, of the subject under study.
The first of these, being rather more difficult to do will be more expensive to undertake, but can potentially deliver a rich seam of information to allow much further analysis to be carried out. The second, involving far simpler collection of mass data will provide huge quantities of data, which may have little depth, that can be analysed using statistical methods.
The assumption here is that you have at least one other solution that could be used for comparison. Qualitative data can be very subjective, making it difficult to understand the questions you really need to be asking. Whereas with quantitative data, the clue is in the numbers. The more you have, the better the overview you should have.
As to what parameters you should measure, this very much comes back to you. Since you have two systems to choose from, I also assume you expect your new system to function better than the old system. If that is the case, regardless of which research method you choose to use, you must still compare one with the other. Using both methods on a single system will give you two different answers. Whereas applying one methodology to both systems, will give you a direct comparison between the two systems.
I assume the old system has some shortcoming, which is why you are trying to develop a new system. You cannot take short-cuts when you want to carry out such research. For qualitative, you need quality data. For quantitative, you need a huge quantity of data. And to compare a new system, you have to have something to compare it against - the old system. So the only answer is to roll up your sleeves, decide what your measurement criteria should be (you should get an idea based on the short-comings of the old system to get you started), and just get on with collecting your data.
From a scientific perspective, if you claim your new system is 50% faster, then you need to measure the data that proves that to be the case. Decide what your questions need to be in order to work out what your measurements need to be. Then you can get started and get the job done. Then you can prove whatever the answers you need are - right or wrong.
I hope that helps.
Regards
Bob
- Badges
- Science topic