Sounds a bit like what is happening in te U.S. My question would be: "Why do we need to measure preservice teachers?" Isn't the university assessment combined with the school based assessments enough? Preservice teachers are on a steep learning curve and the range of imporvement over the initial months of classroom teaching is huge. I have personally seen teachers change from mediocre at the beginning to excellent over the first year and also excellent at the beginning to not much improvement over the first year. Point in time measurment is nearly meaningless in this context.

In the US university system student learning is measured through assessment of learning outcomes and alumni and employer surveys. AACSB and ABET are the accreditation bodies for business and engineering education respectively. Both accreditation processes give a lot of weightage to measurement of learning outcomes.

Thanks so much Mark and Rashmi, in providing your responses, and, in particular, in relation to developments in the US.

Interestingly, one of the understandings emerging is the complexity of answering the question - where student learning improves, who or what contributed to that improvement? That is, what is the supervising teacher or the student teacher or both, or the student themselves, or their parents, or all of them?

Another question emerging relates to Mark's wise insight, that the biggest impacts might not occur and/or be evident and measurable until after graduation when our student teachers become teachers.

In case you are interested, Professor John Hattie, Chair of the Australian Institute of Teaching and School Leaderhip (AITSL) which has responsibility for standards for teachers, provides his insights into the implications and challenge of the Action Now: Classroom Ready report at http://www.aitsl.edu.au/media-newsroom/latest-news/2015/temag-announcement.

As a long time teacher, I am wary of an excessive dependance on measurement. To be useful, measurement should provide a clea indication of future behaviour, not just past behviour. Teaching, learning and subsequent application of learning is so idiosyncratic as to make any measurement more or less meaningless. As well the variables re so many and so misunderstood that even good measurement can be read in a multitude of ways. I know it sounds old fashioned, but ensure teachers have adequate??? training, time and resources to have the desire and capacity to seek to improve and  sensitie intelligent mentors and the system will improve faster than any benchmarked system will. Benchmarking is the Macdonald's resonse. It ensures minimum competence but limits motivation and excellence.

And as an afterthought. I am not aginst measurement per se, just the misuse and overvaluing of measures that don't truly provide the information they purport to. Measurement that is too heavily valued has significnt negative psychological consequences in motivation, self efficacy and self esteem. 

And as an afterthought. I am not aginst measurement per se, just the misuse and overvaluing of measures that don't truly provide the information they purport to. Measurement that is too heavily valued has significnt negative psychological consequences in motivation, self efficacy and self esteem. 

For me, as a champion of recruiting beginning teachers, particularly in international school settings, the measure of success of the initial teacher education program lies in the initial mentoring needed for the teacher to assume full responsibility (effectively) for their first class of students. If the school has to "over resource" the graduate teacher to ensure there is no "slippage" of learning ((whether measured anecdotally or via student (standardised) assessments)). You want the graduate teacher (like the freshly employed doctor) to be hitting the ground running.

To add to Jake Madden's response in mentoring graduate teachers, the 'outmode' for measurement should be on self-selective pedagogical goals which embed student learning. An apprenticeship model for an enabling culture rather than a preventative culture through assessment regimes. 

Of interest to this discussion is that the University of Melbourne has established a "new Teacher Education Effectiveness Research Hub within the Centre for Program Evaluation"- http://education.unimelb.edu.au/cpe.

while I cannot comment on this particular research hub, the phenomenon of establishing such centres lends credence to the act of evaluation before there is adequate research into the mere act of evaluation itself. Many programs/systems have become 'normalised' in the minds of educators in this way. Politicians and their advisors have ideas that spawn research centres  that give them credence before their impact has been evaluated. Not that this is an abnormal phenomenon, just commenting on its negative impacts.

Establishing measurement hubs is an industrial Neo liberal response to a supposed problem with graduate teachers. Surely the best way to "measure" graduate teachers is through a well resourced mentoring program by well tai

This is an interesting question which sits within the broader context of an increasing focus on the effectiveness of public services, target and data driven performativity measures and the metrics of ‘accountability’. Hospitals, schools and many other areas linked with public service are increasingly subject to ‘output measurement’, by means credible or otherwise, as a consequence of policy levers based on ‘league tables’ and ‘naming and shaming’.

Whilst it is reasonable for public services, and the training and supply of those who will work in them, to be subject to scrutiny the real issue is the credibility of the mechanisms by which scrutiny takes place. At the moment this seems rather driven by a ‘one-size-fits-all’ approach of constructing ‘competency’ and ‘levelling’ frameworks which are configured in such a way as to produce ‘comparative data’ which can be processed and used to form ‘rankings’. In the UK this process has been in operation for a number of years and the internal inconsistencies and ‘unintended consequences’ of such performance metrics are beginning to lead to growing scepticism as to its value and reliability. Where agencies charged with making ‘consistent’ judgements about training quality have been subject to multiple and rapid waves of political change the very foundations of credibility are beginning to crumble. The sometimes bizarre consequences of the ‘change storm’ has left us, like Alice, inhabiting a strange world which defies both logic and language. The grading of provision as “satisfactory” no longer  means that it is satisfactory, “outstanding” is expected to be the norm and the definition of “inadequate” is “not good”!!

Comparison between providers of training is made using graded outcomes of student teachers teaching ……… but there are no national criteria for these grades and providers are expected to use their own. Where clarification is sought the phrase “it’s best fit” is offered which, of course, is a weak answer and not one which addresses the concerns of the asker!

The byzantine nature of the system in use is not the result of a clearly thought out system which is a necessary response to the complexity of the task of measuring quality of training but a by product of uncoordinated reform and revision where ideology and political rhetoric has trumped reason, coherence and clarity of purpose.

Inevitably, the shift from measurement of process to the metrics of output has meant that in the case of the initial training of teachers the ‘logical’ and ‘common sense’ approach is to assume a direct and causal narrative linking training -> teaching -> learning. If taken to its logical conclusion ‘the dictatorship of the measurable’ would look towards the grade outcomes of pupils as an indicator of the quality of teaching which in turn is directly related to the quality of training. Of course, such a crude equation does not stand up to the most cursory degree of reason or evidence. We know that teacher MS X will teach a class for a year in school A with one ‘grade outcome’ and then move to another school and achieve a much higher outcome. Sadly, in the UK, this grade-output measure of teacher quality is at the core of ‘performance related pay’. The response to the obvious concerns over this crude approach has been to seek to add further to the metric-madness by factoring ‘value added’ into the equation as some form of ‘statistical fig leaf’.  

I think the UK experience raises some very serious concerns over the feasibility of the construction of a metric for the measurement of ‘training quality’.

  I believe there is some confusion either in the question or in some responses .

  The question speaks of measuring the effectiveness of the Teacher Education , but then goes on to speak of the measurement of the impact of the graduates .  Respondents have come in at a variety of angles .  This is confusing .

  I would agree strongly that people who don't have much of a clue as to what they are responsible for managing try to quantify the variables/outputs involved so as to validate their agendas and/or theories .  They ask for clarity where there is little .  This is a flawed process from the start .  Our common experience is that "They" will take what they want and do what they want - whatever the evidence .

That is the question - because teacher leaders only realise when things went seriously wrong when learners underachieve! Some teachers are lifelong learners, walk into schools, identify their learning needs and their learners achieve. Others, from the same institution, play the blaming game, resist lifelong learning (even at university level) and teacher leaders find it only out when their learners underachieve. Therefore, I would not worry a lot about the efficiency of initial training - I would rather measure a teacher's attitude towards lifelong learning when I have to appoint one ....

 I am very doubtful of what is going on in terms of accreditation, evaluations, quality assurances, and the metrics upon which rankings are based. Are we sure that the best brains in a certain country have met, discussed, and reached conclusions based upon relevant researches into these issues? Have the experienced "sages" of education laid universal benchmarks or points of reference to ongoing measurements & judgments?

When I started school teaching in 1975, an inspector attended one of my classes. After class, he told me things in positive manner but he reported different things in negative manner. I found later that the man was working for his personal ends & not for the sake of betterment of the educational process. In 1976, three experts (one from the United Nations & two from the UNESCO) attended one of my classes & gave me a very positive assessment upon which I was promoted. Sometimes, the educational process is subjected to non-scientific "amateur" approaches which are counterproductive.   

Back to the question...To evaluate Teacher Education Programs I would recommend the following. First, determine what areas of the teaching process need to be evaluated. e.g. Classroom management, curriculum development, differentiation of delivery styles, parent/teacher communication, administration/teacher communication, etc. Then, determine a group of new graduates from each of the universities that will participate in the program assessment. The group should consist of new graduates with no prior teaching experience whom have received full-time teaching positions for their first year upon graduation. Next, determine an evaluation rubric best suited for the population schools that will be assessed. Keep in mind best practices for specifically designed evaluation tools in order to obtain desired information. A goal signifying desired effectiveness for each criteria will also need to be set. Finally, determine a team of evaluator's that possess both academic and professional teaching experience. Have the team evaluate each of the new teacher's identified as part of the assessment group within a few weeks of starting their full-time teaching position, mid-year, and within a few weeks prior to the end of the year. I believe this process would bring to light strengths and weaknesses within various university programs. I also believe the new teacher's would provide an ample amount of qualitative data on what they believe would have been beneficial to have more/less of throughout their program.

As you plan to measure three different components of teacher professional competence as developed through the initial training against the Austrailan Standards, I think you may need to use different instruments. For example, for knowledge , the TKT (test of teaching Knowlelge) may be used,; for professional practice, classroom observations and interviews may be used and the data can be analysed against the Mandated Standards; for Professional engagement, an operationalization of the construct is needed before instruments are designed and used.

Thanks Le Van Canh. I'll explore this instrument. What might emerge is a predictable focus on reviewing a suite of instruments, and, some of these might take on more importance than they should though. This approach can be helpful for some aspects of measuring standards, but can be reductive rather than enabling.

Hopefully, there a development of conceptualisations or frameworks of evidence wich promote a range of other, rich evidence which initial teacher education students might collect, select, reflect and publish through, for example, an ePortfolio of evidence. Your thoughts are appreciated.

Hello!

In some countries this evaluation depends on International Bank trends. Now, we can identify how teacher are evaluating by peers in context through a video. Particularly I consider that this way of evaluation in superficial do not show and cannot help teachers development.