When developing an observer rated scale is test-retest reliability required?

More Danelle Pettman's questions See All

Estimatimating required sample size / effect size with McNemar statistic

I have an effect size from previous research of 1.56 but it uses different statistics, I am not sure how to calculate the required sample size? or the effect size of my findings?

04 May 2017 2,711 2 View

Why are the velocities so high in my Boussinesq approximation Fluent simulation?

Hello, I have a mechanism where air rises due to convective flows. In this simulation, nothing is driving the air except this. I understand that the Boussinesq approximation only works for small...

02 March 2021 4,910 3 View

Has anyone used the Illumina ITS1 primer sets?

Hi everyone, Illumina provides a list of primers to amplify with high taxonomic coverage the ITS1 region for further fungal sequencing, but I cannot find the exact amount necessary of each...

25 February 2021 6,969 3 View

How do you mesh a thin solid in Fluent?

Hello, I have used the shell function on SpaceClaim to create thin solids for the model I am studying. These are contained within an enclosure for my study. I chose this approach as it avoids the...

24 February 2021 8,856 3 View

Am i able to conduct a factor analysis on each of my subscales, as opposed to the whole scale?

My psychometric scale contains 38 items in total, with 4 subscales (20, 6, 6 & 6 items respectively? Can i run a factor analysis for each subscale rather than the scale as a whole for the...

24 February 2021 8,856 8 View

What are alternatives to factor analysis for a small sample?

My sample size is too small to conduct a factor analysis. I created my own psychometric scale for measuring Effectiveness of Support, which contains 38 items (4 sub-scales) Im really struggling...

23 February 2021 9,287 12 View

What is the better software for approximation of 3D surface with equations?

I have a massive of x, y, z coordinates of 2D surface in 3D space, which builded in Ansys HFSS or AutoCAD. How can I get the analitical aproximation of this surface by formulas?

21 February 2021 3,444 7 View

How do I collect online measures in a same-different task?

The same-different task requires to subjects to indicate whether a pair of stimuli seen or heard are the same (say AA or BB) or different (say AB or BA). Researchers often collect offline measures...

20 February 2021 9,226 2 View

How can we assess applied digital skills and soft skills through online examinations during COVID-19 semester?

The final qualification assessment for major programs is a strict regimen that involves different stages - oral and written exams, final project viva, internal and external review. How are ICT...

16 February 2021 1,762 8 View

How many indicators do I need to represent a latent construct?

Hi folks! We did a large survey and planned to use 3 indicators to represent a latent construct (Sense of Place/SoP) in a latent regression model (structural equation modeling). However, one of...

15 February 2021 7,705 3 View

How to calculate the sample size in an interobserver variability study?

Hi I am conducting an interobserver variability study where we have 12 raters who are going to rate samples of lesions. They will rate up to 10 variables per sample. Although the size of the...

12 February 2021 7,276 3 View

Danelle Pettman

Thank you

Muayyad Ahmad

I think you can find suitable answer on reliability testing for observer rated scale in the following link.

https://www.ncbi.nlm.nih.gov/pubmed/19716180

Also, the attached file has more details.

Good luck

Nigel V. Marsh

A generalizability analysis, while more complex to complete than inter-rater reliability, does provide more useful information on how to improve reliability if it is low.

Michael Moore

As Lony observed, test-retest is not applicable here, and the main question is inter-judge reliability.