Dear Mahmood

Your project sounds interesting. You are raising three variables or dimensions of interest: diversity, credibility and impact. I would make the following recommendations:

1) Review the definitions you are using. You are linking credibility with impact. Then you are defining impact in an unconventional way.

2) The literal 3 could be your research question.

3) The fact that you include qualitative and quantitative measures in your study, does not mean that it is of mixed methods. A study of mixed methods goes beyond qualitative and quantitative measures. Mixed methods involve effective integration of these two methods. That is to say if you use the two types of measurement, but not with important integration, your study would be multi-method.

Regards,

Jorge Cruz

Quantitative data includes closed-ended information such as that found on attitude, behavior or performance instruments. Mixed methods research is a research design with philosophical assumptions as well as methods of inquiry. mixed methods usually refers to contexts in which a researcher collects, analyzes and integrates both qualitative and quantitative data within a single study or longitudinal program of inquiry. The mixed methods researcher, pragmatism opens the door to multiple methods, different worldviews and different assumptions as well as to different forms of data collection and analysis in the mixed methods study like as Sequential, Concurrent, Transformative, Quantitative Experimental designs

To me, quantitative research involves formulating hypotheses, creating measures of key variables to operationalize those hypotheses, and testing the hypotheses in terms of the relationships among the variables.

Mixed methods would combine that form of research with qualitative methods that could serve purposes such as exploration and interpretation.

Dear Jorge,

Thank you for your answer. Let me elaborate it bit further.

First Data set will involve information sources in term of diversity (that involve numbers)

Second data set will assign a value to each source on the basis of its credibility (i.e. international reputation of the source)

Both these data sets will then be linked to organizations’ standing in the international rankings.  

In nutshell,

1.      What is the diversity level in sources consulted?

2.      Are these sources credible?

3.      Is the any relationship of 1 & 2 with organizations’ standing in the international ranking?

Thanks again for time and suggestions.

I really don't see anything qualitative here -- just different degrees of the extent to which you use subjective or objective means to measure things.

Dear all, thank you so much for your valuable suggestions. I am indeed grateful to you. 

Dear  Mahmood Ahmad

Your research have a proposefull  sample that seems more appropiate quantitative methods. If you use qualitative data,the mixed method  will be "more quantitative than qualitative"!

I fully agree with the opinion of prof. David L Morgan and Jose Contente ·  

Good luck!

I think David Morgan captures the key points in his comments - it reads as a quantitative approach.

Hello Mahmood,

I would like to offer you advice about a different kind of thing from the above contributions. 

I notice that you are using impact factors of journals as a measure of their respectability or of the likely quality / credibility of submissions made to them.  In my experience, impact factors can be misleading for a variety of reasons - and I have seen a lot of rubbishy articles in journals with quite high impact factors and high quality articles in journals with low impact factors. So there is a risk that you might engage in quantification that is not well founded. 

This is difficult, I know. I wonder whether the h-index might be a more valid indicator of quality of publications.

All the best with your research.

Robt.

Dear All,

Thank you so much for the overwhelming response. I am indeed grateful to you all. Now it has been cleared that I would be using quantitative method (or multi-method).

However, I need a little help or any suggestions for further refinement and improvement.

There are three variables that are: Diversity, Credibility & Impact. These are defined below:

Diversity is described mainly in publication type (Journal’s articles, newspaper’s article, books, websites, official documents, etc.);

Credibility in terms of the frequency of specific type of sources (scholarly or peer review and non scholarly – i.e. Scholarly article is more credible than say a newspapers article – since there is no well defined scale to weight every source (or at least I couldn’t find in the literature) so I think these two main categories will be appropriate, however, any help/suggestions in this regard will be highly appreciated)

Impact in terms the relationship of both the above variables with think tanks' standing in international ranking (i.e. Does high rank think tank uses diverse and credible sources?)

I would appreciate any help or suggestions to include or exclude anything from the above.

Thanks again!

Hello again Mahmood,

I notice that you are intending to assess credibility as a dichotomous variable.  It would be very easy to do so in the way you propose ("scholarly article" versus things like newspaper articles) - but in doing that you might be creating a very blunt variable - particularly because many journal articles look impressive on the surface, but with even minimal analysis turn out to be misguided or shallow.  That is not to deny that there are many  journal articles that are impressive at all levels - but most meta-analyses provide evidence that a lot of research can be dismissed as inadequate.

On top of that, I've seen some very impressive material in newspapers.

In essence, I think that many journal articles are not very credible, and many newspaper articles are.

I'm not sure what you can do about this (I suspect you're correct that there isn't a well-defined "scale" or means of assessing credibility), but if you want your results to be meaningful, I would encourage you not to use the kind of dichotomous variable for credibility that you are proposing.

I hope this is helpful.  Defining variables, and measuring them appropriately, is at the heart of much quantitative research.  And the task can produce headaches!

Robt.

Thanks a lot Robert for your insights. I am totally agreed.

However, as I said that there isn’t an all out criteria for judging the credibility of every source and even if there is any that would be subjective.

Moreover, since I have taken a sample of the reports of top 50 think tanks over one country in the last 10 years. Keeping that in mind it would be a gigantic task to determine credibility by looking into the contents of every source. 

What I intend to do is to give some idea based of established norms (at least scholarly or peer review research has gone through certain process and reviewed by people who are experts in their filed).

I wonder if this is insufficient or inappropriate what other factors I should be looking at to determine the credibility of sources. Any suggestion in this regard would be highly appreciated. 

Hello again, Mahmood,

I'm glad that you appear to realise that I want to walk with you in this, rather than to throw up problems that are insurmountable.

My responses might be a bit inadequate because I'm not familiar with the full context of your research.  If so, please take that into account and forgive me if necessary.

If one of the issues that you want to investigate is whether the "traditional" categories of what is credible or not are related to other variables, then a quite basic division of (conventional) categories concerning credibility might work quite well.  You might, for example, find out that the traditional categories of respectability / credibility are inadequate or are not related to other important things (which is the point that I am driving at).

I am concerned that you seem to believe that the (peer) reviewing process used by journals guarantees their credibility.  My exposure to research in a variety of fields (including psychology, health sciences, policing, education, and mathematics - just for some) has convinced me that the reviewing process for journals is not all that it's cracked up to be.  The reviewers are obviously often not, as you say and as is usually touted, experts in their field.  I suspect that they are often time-pressured people who get the task of reviewing off their desk as soon as they can possibly squeeze a bit of time in to do it.  In fact, I have just submitted an article to a journal in which I criticise four articles for a range of things from being ill-conceived to the researchers using the wrong statistics and interpreting their results dishonestly in order to maximise the chance that their research will be published - and the reviewers and editors obviously fell into the trap.

I also have a colleague who strongly believes, I think too cynically, that journal articles in her field are published not on the basis of their quality but on the basis of whether authors are part of the "club" of editors and reviewers at the journals - and, as a result, poor quality articles are often published within those journals.

So, I'm not sure where this gets us.  It's difficult.  As far as I am aware, there are no valid means of assessing quality of publications - and there are debates about how (in)adequate things like impact factors and H-indices are.  Some of these debates are readily available on the web.

My main concern on your behalf is that you don't naively regard things that are published in journals as having high credibility. Maybe it would be better to conceive of them as having overt / conventional credibility, but not necessarily "real" credibility.

I am also concerned that you might be attracted to quantification that is not well founded. I very much like quantification, but I've seen too much silly quantification to support quantification unthinkingly.

I hope that's helpful.

Robt.

It is not researcher's choice to use or select a method or strategy rather its research questions guides researcher which techniques provide an accurate answer. Therefore, instead of asking other, understand your research questions and search which technique will provide the best answer.

It is not the patient decision which treatment he/she needs rather it disease guides doctor which treatment is best for the patient to cure disease and save life.

Dear Mahmood.

You already mentioned in your question:

I am still not very much clear whether it is pure quantitative studies or mixed method research. Any help would be highly appreciated. 

The clarity comes from research questions. Mixed method is always in those cases where there are multiple questions and for each question, a single method is not appropriate.