Hi Alice, there is a tool called the Counselling Self-Estimate Inventory (COSI) that has been shown to be a vaild and reliable instrument for self-rated measures of counselling efficacy. See: Larson, L.M., Suzuki, L.A. et al. (1992). Development and validation of the counseling self-estimate inventory. Journal of Counseling Psychology, 39, 105-120.

Also see Beitman, BD and Yue, D (2004),. Learning Psychotherapy: A time-efficient research-based outcome-measured psychotherapy training program (2nd Ed.). New York: Norton & Company

Hi Alice, several relevant measures are discussed in the following:

Urbani, S., Smith, M.R., Maddux, C. D., Smaby, M.H., Torres-Rivera, E., & Crews, J. (2002). Skills based training and counseling self-efficacy. Counselor Education and Supervision, 42, 92–106; Lehrman-Waterman, D., & Ladany, N. (2001). Development and validation of the evaluation process within supervision inventory. Journal of Counseling Psychology, 48, 168–177; Green, H. J. (2011). Skills training and self-esteem: Educational and clinical perspectives on giving feedback to clinical trainees. Behaviour Change, 28, 87-96

Many thanks for everyone's responses - where can I get a copy of the COSE? I have the article listed, but am not sure of where to purchase the COSE or how to mark it.

Again, thanks.

Hi Alice,

we started to develop something simple but are still in progress with assessing it and it was based on the current curriculum rather than the new CBFP.. Happy to share what we have but it hasn't gone through psychometric property evaluations. Maybe that's a good PCP project for someone? The first is around the curriculum competencies and the second asks more detailed info about specific therapy types. Cheers, Loy

Here is the Part 1 Alice. Also I very much agree with the scepticism about a trainee's self assessment. In the Westmead program we do find the taping and the trainee's developing capacity to reflect on the audiotaped sessions is an invaluable teaching and assessment tool. The Conversational Model is currently developing a scale to look at adherence to the model as again it is the inter-rater reliability and other properties that are the challenge.

Loy

Thank you for sharing these documents. While not a psychiatrist, I do educate music therapists, and am always looking for ways to assess their skill. In my experience, the music therapy students typically rate their abilities lower than their clinical supervisors. Not sure if this is true in other disciplines. I am intrigued by Patrick's suggestion that patient ratings of skill level can predict outcomes. I am wondering if there is a study I can read on that topic.

Hi Loyola,

A big thanks for these documents.

Alice

I am intrigued by the topic and appreciate all contributions, literature, and, particularly, Loy's willingness to share his research project with us by making documents available to us! I am a doctoral student in Clinical Psychology and, if you will, sit on both sites of the table, i.e., I sort of assess my patient in different way and get evaluated by supervisors *a lot* (I currently have four different supervisors for both psychotherapy and psychological assessment patients). I have experienced how self assessment and view of self by others can diverge drastically.

Thanks Alex, as a Psychotherapy Educator, this discrepancy is an ongoing issue. we do find that trainees often profit from some group work where they can hear each other's auditotaped work as this often helps them "get it" when they hear others and sometimes when they have the experience of peers admiring or enjoying their work. But just for the record, I am a she!

Thanks, Loy,

Yes, agree. This is a very important yet under-researched area. It is helpful to audio - and videotape. I am working on a study that involves student clinicians' assessment of countertransference feelings, evaluated and measured through questionnaires and coding of videotaped sessions. Now, I wonder whether it would be beneficial to include measures of self-esteem in the study...:-)

I am a she also (Alexandra)

Thanks Alex.

I think the self-esteem issue is very interesting as we know it can influence appraisal profoudly. As an attachment researcher it also raises for me the notion of the trainee clinician having some sense of their own attachment strategy and how it might combine with the patients to influence countertransference. Those attachment templates do interact strongly with self-esteem as well.

My pleasure, hope it helps.

Dear Alice,

I have in couple of papers (published in Psychotherapy Research) examined the effect of psychotherapists' self-assessment on early process and on patient outcome in naturalistic psychotherapies with ca 300 patients and 70 therapists, using the mulitfaceted large-scale questionnaire Development of Psychotherapists Common Core Questionnaire (Orlinsky et al., 1999; Orlinsky & Rønnestad, 2005), I have published both papers on my ResearchGate site, for you to download and maybe they are of relevance for your purpose? Best regards, Helene Nissen-Lie

While I do not have a measure specific to your context note Bandura indicated that really the self-eff scales/items need to be developed pretty context specifically. You may be able to adapt something from one we developed for group leaderships skills a while back (it is downloadable from my pubs I think).

Development of the Group Leader Self-Efficacy Instrument. Betsy J. Page, Dale R. Pietrzak, Todd F. Lewis, The Journal for Specialists in Group Work 06/2001; 26:168-184. DOI:10.1080/01933920108415736 pp.168-184

Bandura and others working in self-efficacy have questioned the validity of a general sense of self-efficacy. On this account, they proposed that self-efficacy scales must be constructed individually for each area of performance. One does not have to do this blindly, of course. There are any number of self-efficacy scales available in the literature, for this construct has been thoroughly research over the last 30 years or so. These scales can serve as models for constructing a scale for assessing self-efficacy in psychotherapy if one does not already exist.

After working with a number of very experienced, senior basic and applied science social psychologists I learned from them, and fully agree those who recommend that that self-efficacy scales must be individualized, made specific to the very particular skills or behavior. For example, consider all the skills and new behaviors one must learn and be able to perform with confidence and competence in a general form of CBT, cognitive behavior therapy, with a single construct within it, such as challenging/questioning of fixed false beliefs. Here, the therapist must acquire the skills of:

1. identifying a false belief

2. determining whether this particular false belief is suitable for reframing

3. determining whether this particular belief is the most important false belief to address with the client

4. determining whether working on this false belief at this time in your work with the client is the right, best next step, vs other alternatives, such as relaxation, guided imagery, exposure, problem-solving an area of the client's life, etc.

5. knowing how to raise this fixed false belief with the client so that they can readily agree that their belief is incorrect, and that this recognition occurs as a powerful emotional realization, rather than as simple cognitive awareness

6. motivating the client to experience a strong need to challenge and change this belief

I can think of 5-10 more skills and new behaviors trainee therapists must learn, even in this single construct.

As a result, the task of the teacher, and then of the researcher, is to identify the most important skills the trainee MUST learn and master to achieve competence, i.e., they must identify all of the particular, CBT skills and the associated behaviors, one-by-one. Each of these skills (and the list may get way too long) can be associated with the trainee's self-efficacy beliefs, each of which can be measured on as a single item often on a scale 0-10.

For example:

How hard/difficult or easy is it for me to identify clients' false beliefs that are ideal candidates for questioning/challenging?

Very Very

Hard So-so Easy

0 1 2 3 4 5 6 7 8 9 10

In the example, a therapist may have superb skills and high self-efficacy in 5 of the 6 skills above, but end up doing poorly with the client because of problems in only one of these skills.

Also, when one begin with new trainees, they often have high self-efficacy beliefs, thinking they are "good" at something, only to realize they didn't know something nearly as well as they thought, and their self-efficacy drops...as it should.

Global self-efficacy questions tend to over-estimate self-efficacy at the individual behavioral level. Thus, a global question such as

"How hard/easy is it for me to apply the specific technique of 'challenging or questioning false beliefs' when doing CBT with clients?" is likely to be not nearly as useful as more specific questions.

Taking this point to the extreme, it is easy to see that an overall self-efficacy question "How hard/easy is it for me to do CBT with clients?" will be nearly useless.

I'd welcome other points of view.

This is an update more than an answer. We are starting a small QA project looking at how Balint groups improve psychotherapy skills of empathy and self-reflection in our psychiatry trainees. We have modified the Psychotherapy Skills Questionnaire I shared with Alice by adding several probes and shortening other material. We will keep you posted. We intend however to back this up with a fuller research program including interviews that can be scored qualitatively by thematic analysis and discourse analysis and ideally want to be able to look at RF and metacognitive markers (the latter is work in planning).