Hello everyone,
In my research I will be assessing the relationship between a number of psychological variables , the issue I am having is that I could not find a suitable tool to measure one of the variables and I had to design a one myself and I am a bit worried as I will measure the other variables using well established tools. I have never come across a study using both existing and self-developed tools and I am afraid this will undermine the methodological rigor of my research.
Hi Ouissem,
You can use self-designed measures with existing validated measures, for sure.
The issue is how you can show that your own measures are reliable and valid. Ideally, you would collect sufficient data in the study to assess the validity and reliability of your new measures while also conducting the actual study of interest. Ideally, you should ensure that you fully cover all facets of the construct you are interested in, use more items in the measure than you think you will require (just in case some do not work out as planned), and include other measures that are known to correlate with your construct of interest and some measures that are expected not to correlate with the construct. Thus, you can assess the factor structure (exploratory factor analysis is the sample size is sufficient), internal reliability of retained factor (e.g., Cronbach's alpha), and then use the other measures to assess expected correlations for validity checking.
This is a bit of work, but it will increase the robustness of your study substantially. Potentially an initial manuscript can be written covering the development of your new measure.
Cheers,
Royce.
Royce Willis has given you could advice. In addition, you might consider pre-testing your newly created measure. One possible technique for doing that pre-testing is through cognitive interviews (Willis, 2004).
Besides the good insights above, another approach to pretest interviewing for questionnaire development is the Qualitative Pretest Interview (QPI) by Buschle et al. (2021). QPI aims to minimize response and measurement errors ensuing from problematic survey items. Here is the full citation.
Buschle, C., Reiter, H., & Bethmann, A. (2021). The qualitative pretest interview for questionnaire development: Outline of programme and practice. Quality & Quantity. https://doi.org/10.1007/s11135-021-01156-0
Article The qualitative pretest interview for questionnaire developm...
Good luck,
In addition, you could go through the book by Willis (2015) for updated inputs on cognitive interviewing. Here is the reference.
Willis, G. B. (2015). Analysis of the cognitive interview in questionnaire design. Oxford University Press. https://global.oup.com/academic/product/analysis-of-the-cognitive-interview-in-questionnaire-design-9780199957750?cc=us&lang=en&
Good luck,
For a single variable or parameter, you can measure it via your tools, then compare it with the existing one.
- Badges
- Science topic
- Similar topics
- Psychology