You could trace the paper's citation behaviour through various scholarly databases. An influential study will probably have been cited somewhere, and the citation destination would also give you a good clue on whether to use it or not.
Aside from that, the journal itself must be indexed in some database as evidence of quality assessment. Journals usually have a list of these indices visible in their profile.
Thank you very much Maria for your answer to my question. As I see it has been cited in papers which where not published in indexed Journals. This reduces the value of its paper i guess. Am I right?
I think if the study fulfils your inclusion criteria, it should be used whether or not the study was from a journal with no IF. The aim of a systematic review is to summarise all relevant individual studies. These studies can come from journal articles, conference abstracts, book chapters etc. If the study fulfils the inclusion criteria, you can evaluate the quality of that study through a scoring system. Then run sensitivity analyses.
Thank you Carmen for your contribution to the question I have. Could it be possible for you to mention any particular scoring system which would be proper to estimate a paper?
I am not sure if you are dealing with randomized or observational studies. If you are dealing with observational studies, you can use the Newcastle-Ottawa scale, ROBINS-I etc. Bear in mind that there is no one correct scoring system or perfect scale to assess the quality of the studies that you found.
For RCTs, you can check :
Article Assessing the Quality of Randomized Controlled Trials: An An...
Of course you can use, there are many good journals with no impact factor or very less impact factor. The main important thing is to look over the quality and where it is indexed in the database or not atleast in google scholar if not in pubmed or scopus.
Meanwhile I also understand that there are many good print journals, in which some of them may not be indexed anywhere online.
so, it's really a debatable issue how to assess the quality of paper, So as suggested by Carmen if the study is clinical trials you can go through that scoring procedure.
But what about other type of studies, it's really debatable.
Even if you are going further for meta analysis, quality of the articles play a major role in your outcome, so in that case it should be handled with more care.
Reading the Newcastle-Ottawa scale I see that it is mostly utilized when it comes to review or meta analysis of medical issues. If this is true what about a tool regarding issues related with sciences as Marketing. Is there any particular tool to be used in that discipline?
IF is not necessarily the sole indicator for paper quality. I have come across non-indexed journals (at least in the JCR list) that host remarkable research. Your inclusion criteria is probably more important than these metrics, even more so if you take into consideration the quality of references used.
I have an additional question: Could we use Google Scholar as the searching criterion for the collection of research papers? I am a HR research scholar and am doing systematic review on soft skills.