As an editor assistant, I frequently encounter conceptual papers and suddenly feel that I do not have sufficient devices to asses them appropriately.
Is there any "rule of thumb" or "guidance" that can help me to assess whether a conceptual paper is of quality or not
I have to assess conceptual papers occasionally as a reviewer and more frequently as a reader. My assessment practice is not so different from the assessment of empirical papers. My first question always is: Does the paper raise a relevant question? Does it address a gap in our (in this case, theoretical) knowledge? Since I mostly do empirical research, one of my criteria for relevance is the impact of the question and answer on empirical work. Although I admit that there might be papers whose relevance is purely theoretical, I tend to prefer those that support empirical work (e.g., by clarifying concepts and relationships between them).
This rules out most papers that just discuss differences between theoretical approaches. It also rules out papers that propose a new theory to solve a problem that does not exist outside this particular theory. In the latter case, nothing is lost if the paper is not published :).
If the paper addresses a relevant theoretical question, my second criterion is the quality of the answer. Does the paper really solve the problem it poses? Is the argument theoretically consistent? A common problem in this respect is that answers combine theories that rest on contradictory ontological or epistemological premises.
I hope this helps.
How to assess a conceptual paper (social science)?
Not sure there is any rule of thumb how to evaluate a conceptual paper. But you can review the manuscript in entirety & try to score each attributes accordingly. E.g. you can refer to the following template. Inside the template have 14 attributes - you can exclude those attributes not related to empirical evidence (since this is merely conceptual paper) e.g. methodology, result findings etc. then set your minimum passing score for evaluation. Wishing you all the best.
Research Manuscript Review Template - Feel Free to Download
I have to assess conceptual papers occasionally as a reviewer and more frequently as a reader. My assessment practice is not so different from the assessment of empirical papers. My first question always is: Does the paper raise a relevant question? Does it address a gap in our (in this case, theoretical) knowledge? Since I mostly do empirical research, one of my criteria for relevance is the impact of the question and answer on empirical work. Although I admit that there might be papers whose relevance is purely theoretical, I tend to prefer those that support empirical work (e.g., by clarifying concepts and relationships between them).
This rules out most papers that just discuss differences between theoretical approaches. It also rules out papers that propose a new theory to solve a problem that does not exist outside this particular theory. In the latter case, nothing is lost if the paper is not published :).
If the paper addresses a relevant theoretical question, my second criterion is the quality of the answer. Does the paper really solve the problem it poses? Is the argument theoretically consistent? A common problem in this respect is that answers combine theories that rest on contradictory ontological or epistemological premises.
I hope this helps.
Thank you very much Jochen Glaser , Mauro Luisetto , and Han Ping Fung
Your explanations are very useful for me
I am agree with Jochen Glaser. My suggestion is: you need to be a good knowledge expert doctor for judging/reviewing the work of other researchers. You need to have strong and very up to date knowledge in the relevant area is a key to be a referee, otherwise you will put wrong injection and the patient (candidate) will get in big trouble.
In summery again, your good training (previous+new = Literature reviews) and strong background knowledge in the relevant subject/area is "rule of thumb" or "guidance" to grade new research work.
- Badges
- Science topic
- Similar topics
- Social Science