My first question would be to ask why factor analysis is not possible?

Beyond that, you could use coefficient alpha with the option "alpha if item deleted" to remove items for each of your three categories.

Dear professor Morgan,

Thank you for your answer

I think that using factor analysis is not prohibited if the conditions are ready, but it is not the best method of choice in formative tools, because unlike reflective tools, in these type of tools, our goal is not to reach the high correlation between the items and the reduction of the items; in fact, with the necessity of the item being determined by the opinion of the expert panel after the content validity, what is the necessity to reduce the number of items that increase our accuracy in examining the desired construct? It seems that it is preferable to use convergent, divergent, and discriminant validity in the case of formative tools.

A lot depends on the circumstances in which you are going to collect your data. If you are asking people to complete a 42-item set, that can be demanding, so it would make sense to remove redundant items.

Yes, you are completely right, in a reflective tool, that is self report one, and we ask others like patients to complete it, we can remove redundant items, but no in formative ones, that will fill by coders, raters or observers like GCS, APACHE,TEAM, and so on

You are also right about the number of items that can be adjusted depending on the situation. Our tool is designed based on the review of resuscitation videos in a simulated environment and is most suitable for these environments and educational purposes, although rating recorded videos of real resuscitation can also be done . Obviously, it is better to reduce the number of items if we decide to use the tool in the real resuscitation setting.

Constructing an instrument using formative indicators is constructing an index which for instance can be used as a predictor for some effect.

With reflective indicators the measured concept is causing the indicators, with formative indicators it is the opposite. The indicators determine the concept. Therefore, this is, in fact, not measurement.

The Apgar score for predicting health problems with a newborn is a good example of an index which is based on formative indicators.

The reflective model is a series of regressions, each having an indicator as the dependent variable and the latent construct(s) as independent variable(s). The ensemble of these regression are then used for estimating the construct(s). This is what you do with CFA.

In a formative model, you have the construct/latent variable as the dependent variable in one regression model with the indicators as independent variables.

Therefore you can use an ordinary regression analysis to show which indicators contribute significantly to the model - while taking care of the usual problems in regression analysis, like collinearity.

In your case it seems that you want to evaluate three aspects of the collaboration of the teams, therefore you should probably use three regression models, each with the appropriate indicators as independent variables, and then you find which of these contribute significantly to (the variance of) each aspect.

These analyses can be used for determine which indicators to retain. It could also be used to construct weights for the indicators to construct the three indexes to use for evaluation and prediction. However, it would require large samples and maybe also an incremental research process to estimate reliable weights

Since this seems to be the explorative phase of the research you might divide you sample - provided it is large enough - in two in order to use one part to test the performance of the models developed using the other (which possibly ended with fewer indicators by excluding non-significant ones).

However, you probablly should not base the development exclusively on the statistical results if you find some indicators important to retain, despite being non-significant in your statistical analyses.

Dear Dr. Ivanouw,

Thank you for your valuable answer. The formation of three categories is based on directed qualitative content analysis. In our study, we do not seek to design a model and predict the result, because, in resuscitation, the most important factor related to the outcome is the severity of the disease. The goal was to develop a valid and reliable tool to identify the strengths and weaknesses of teamwork in resuscitation. I agree with your opinion in the last paragraph that, in formative tools, one should not rely only on statistical tests.

I understand that your goal is to develop a reliable tool. In order to do that I understand that you have performed qualiative content analysis. However, I am not clear about the data for these analyses. Do you use observations of team work, or perhaps questionnaires for the coworkers? Whatever, These data then may be the indicators you want to use for identifying that strengths for the teamwork?

If this is the case, you in fact are using the principles of regression analysis, namely to identify which indicators/data best evaluate the strength/weakness of the teamwork.

Am I right?

Best

Jan Ivanouw

Yes, the data collected based on scale items are used to determine the strengths and weaknesses of teamwork in resuscitation.

As you know one principle of regression is a term used by real estate evaluators stating that the value of high-end real estate may be diminished by having lower-end properties in the same vicinity. If you mean this principle, regarding the first step of regression analysis (analyzing the correlation of the data), as mentioned before, in formative scales, we do not look for the correlation of items, even the correlation between items within the same category like leadership in our scale.

As you know, in regression, the relationship between a dependent variable and several independent variables is usually examined. We do not have dependent variables in this case.

There are even other formative tools that, although independent and dependent variables can be considered for them, there is no need for correlation among the items of each category. Because there can be positive, negative, or even zero correlation between category items in formative scales. For example, there are varibles in physiology variables category of APACHE II score, that have no correlation between them like temperature and serum Na. In these cases, regression does not help, please advise if my understanding is wrong.

Best regards

I am not sure our understandings of regression analysis are the same. Just one remark. The dependent variable would be each of the three categories and the independent variable would be the items you are using for evaluating/determining these categories

I think that the difference of opinion is related to our different views of reflective and formative tools. In formative tools, the objective of the study and the opinions of experts are much more important than statistical tests. categories and items of many similar tools, such as ANT, and OSCAR, have been determined solely based on the review of literature or other similar tools and finally with the opinions of experts during determining the validity of the content.

As you know, even in factor analysis, experts play an important role in removing or keeping items or the categories in which items are placed, and statistical methods and tests are used only where they are justified.

In our study, the data obtained from integrative review, review of related tools, in-depth semi-structured interviews and observations in the field, and determining the face and the content validity by 12 expert panel members resulted in a 44-item tool whose convergent and criterion validity was also confirmed.

Let me explain the subject better with an example of a formative tool. In standard electrocardiography, we use 12 leads to examine many heart-related issues. Does 12 leads show the situation better or 8 leads? Is the lower part of the heart, which includes leads II, III, and avf, considered a dependent variable in relation to the three leads? Should there be a correlation between the leads of the lower area? Can we change the place of the precordial (v1-v6) in different places and determine the regression between them or use factor analysis? In the development phase of a well-known tool such as APACHE II, which has the data of thousands of patients hospitalized in the ICU, the main categories that determine the severity of illness, such as the selection of vital systems of the body, were not based on regression or factor analysis. the authors used regression tests to reduce the number of indicators related to each categories have been determined.

The purpose of this study at this stage is to determine the teamwork of the resuscitation team members, whether in a simulated environment or by examining the videos prepared from the real environment, in order to determine the strengths and weaknesses of the teamwork in more detail.

Thank you for your explanation. In order to better understand could you tell what is the 3 main categories and (some of) the 44 items?

Mohammad Hosseini It sounds like you have no intention of modifying your tool, so I have to wonder why you posted this question in the first place?

you're welcome, sure

The qualitative part of this mixed method study was larger and the number of items extracted from the interviews made up the majority of the item pool. According to the results of the qualitative phase, a number of environmental and member-related factors also had a great impact on teamwork, and this group of codes formed the first category: "essential prerequisites for resuscitation". The next category was leadership, and the third category was teamwork. Teamwork in the title of the scale is more general and includes all the factors affecting the teamwork in resuscitation.Teamwork as the third category is more detailed and only includes the interactions between the resuscitation team members. In the following, three questions are presented for the first to third categories in order.

1- Team members and leaders adhere to the orders.

2- Team members and leader are capable of performing clinical skills.

3- Team composition is specific, standard and consistent.

4- The team leader adheres to the treatment protocol with time management.

5- The team leader clearly shares information and goals with team members..

6- The team leader organizes and coheses the team.

7- Team members seek to avoid or contain error.

8- Team members are coordinated in doing things.

9- There is an atmosphere of cooperation between team members.

This tool comes with a training guide and the raters must be trained. In the training guide for each item, examples of weak and strong teamwork are also given.

Dear Professor Morgan, Thank you for your question,

This week, Wednesday, the final defense of my PhD thesis is scheduled to take place. Two weeks ago when I talked with one of the known professors other than the research team , he suggested to do the item reduction after doing the face and content validity. It seemed to me that because most of the existing tools are reflective and factor analysis is used to reduce the number of items, the professor recommended using this method. It was suggested that the number of items be reduced with the Fleiss Kapa coefficient or another method. Fleiss Kappa coefficient is related to inter-observer reliability, how can one question the validity of a tool that has been confirmed by content, convergent and criterion validity with low reliability? Another suggested test that I found for ranking data was Kendall's w coefficient, which is also use correlation, while correlation between items is not very important in the formative tool. After further investigation and studying the differences between these two types of tools, I realized that there is no reason for doing this; because I did not find any information about the necessity of doing this advice in formative tools after doing the content validity, I decided to publish this question. To ask the opinions of other professors.