Hi,

It seems that multiple choice questions are the opposite of creative thinking. One is given 4 choices of answers to a question, but only one is the correct one. So, in fact, this is a closed system. In order to be creative an open system is needed. The correct answer was established by others, so you do not need to make an effort to build an answer. This effortless answering method does not indicate creative thinking.

Creativity starts with imagination. In MCQ, there is no space of independent  imagination, therefore it is not a good method of measuring creativity.

Hi Vigai!

You ask if it is possible to measure 'creative ability' through Multiple Choice Tests. I would say: Not certainly. Like Gordon and Ranjan, I would even say that creativity is at odds and at complete variance with multiple choice tests. Note that creativity is above all divergent and critical thinking, whereas multiple choice tests are, as it were, closed systems. You could not go beyond one of the 4 presented alternatives, nor do you have to justify for the chosen alternative. In what follows I elaborate a bit on the reasons why try to assess creativity through multiple choice tests is tantamount to wanting, say, to contain the Atlantic Ocean into a glass of water. Impossible! 

It is generally alleged that critical thinking and creativity refer to one’s capacity to think clearly, rationally, and divergently about what to do or what to believe; that it includes one’s ability to engage in reflective and independent thinking; that peoples with critical thinking and creativity skills are capable of, for example, understanding the logical connections between ideas, identifying, constructing and evaluating arguments, detecting inconsistencies and common mistakes in reasoning, solving problems in a systematic way, noting the relevance and importance of ideas,  and reflecting on the justification of one's own beliefs and values.

Critical thinking and creativity, therefore, are not a matter of compilation of information. A person with a good memory and who knows a lot of facts is not necessarily good at critical thinking and creativity. A critical and creative and innovator thinker is able to deduce consequences from premises s/he knows, s/he knows how to use information to solve problems, to seek important sources of information to inform himself/herself and to invent solutions for solving problems that are new, original, and innovative (see below the F. Gauss' example).  

A common misunderstanding about critical thinking and creativity is to judge them  as being critical of other people and, say, a given of nature. Although critical thinking skills can be used in exposing logical fallacies and bad reasoning, critical thinking can also play an important role in cooperative reasoning and constructive tasks. Critical thinking, therefore, can help us acquire knowledge, improve our theories, and strengthen arguments. Of course, we can use critical thinking and creativity to improve social institutions. Note also that creativity can be fostered and is not necessarily pre-determined in one's brain. It suffices to say that maturation and genes open up a series of possibilities, but without giving necessarily rise to their actualization. More to the point, is the person as a whole, not his/her brain who is critical and creative.

Other common misunderstanding about critical thinking is that it hinders one’s creativity because it requires following the rules of logic and rationality, and creativity might require breaking rules. Critical thinking, however, is quite consistent with thinking, say, "out-of-the-line", challenging consensus and pursuing divergent ideas and less popular approaches. If anything, critical thinking is part and parcel of creativity because we need critical thinking to evaluate and improve our creative ideas. 

The following example clearly illustrates that creativity or divergent thinking and critical thinking are highly consistent with each other. It is said that Karl F. Gauss (1977-1855) left his primary school teacher highly perplexed when, at the age of 8 years, he gave a creative answer to the following problem his teacher had written shortly before on the blackboard: “ What is the sum of 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8?”. All of a sudden, and apparently without having time enough to perform the respective operation of adding, Gauss replied that the result of the arithmetical operation at hand was 36. “Why is this so?” -- The teacher asked again. She became more perplexed when Gauss replied that this was so because 4 X 9 = 36. “I cannot understand” -- The teacher replied and went on:  “Why did you perform an operation of multiplication instead of an addition operation?”  “I did that -- Gauss replied -- because I easily realized that 1+ 8 = 9; “2 +7 = 9; 3 + 6 = 9; and 4 + 5 = 9. Hence, 4 X 9 = 36”. Even if it were given by a non-expert adult in mathematics, we would certainly say that Gauss’ answer and way of thinking was highly creative and was a clear example of divergent, new, and insightful thinking. As critical thinking also includes one’s ability to engage in reflective and independent thinking, Gauss’ answer is telling example of both creativity and critical thinking in mathematics.

The example also substantiates a procedure of how to get access to one’s critical thinking and creativity in mathematics, physics, psychology, education, and so on. In other words, to ask individuals to justify their answers or performance on issues wherein critical thinking and creativity may be involved is a kind of “think aloud” procedure, a procedure often employed by researchers to get access to underlying psychological processes at issue, for example, in one’s critical thinking and one’s creative thinking or acting. Nothing similar to this happens in multiple choice tests. 

Critical thinking and creativity often involve evaluating arguments and detecting inconsistencies and common mistakes in reasoning. Because of this, the domain of conditional reasoning is an appropriate domain for one to observe individuals’ ability to think critically and in  an creative way, that is, to see, for example, how individuals of different ages solve each of four classical logical arguments (i.e., Modus Ponens, Modus Tollens, Denial of Antecedent, and Affirmation of Consequent). Of course, for one to know individual’s critical and creative  thinking while solving these logical arguments or problems or other problems one has to ask them to justify (or think aloud) their responses and solutions.

As you know, conditional reasoning has do to with “if…then statements”.  Modus Ponens: If p (Mary is at school), then q (John is also at school). P is the case, therefore q is  necessarily the case (Mary is a school). Modus Tollens: If p (Mary is at school), then q (John is also at school); not q is the case (John is not a school): Hence, not p is necessarily the case (Mary is not at school). Denial of Antecedent:  If p (Mary is at school), then q (John is also at school); not p is the case (Mary is not at school). Hence, nothing can be concluded about q (i.e., John may be or may be not at school). Affirmation of Consequent: If p (Mary is at school), then q (John is also at school; q is the case (John is at school). Hence, nothing can be concluded about p (Mary may be or may be not at school). There is mounting evidence that shows that even 5-year-olds draw the “correct” conclusion when Modus Ponens is the case (i.e., John is at school), that 9-year-olds draw the “correct” conclusion when Modus Tollens is the case (i.e., Mary it not at school), and that even some adolescents and adults are not capable of drawing the correct conclusion when Affirmation of Consequent and Denial of Antecedent are the case (i.e., It is not possible to say whether or not Mary/John are at school). Of course, children are not capable of dealing with either Affirmation of Consequent or Denial of Antecedent problems. It is worth mentioning, however, that when 9- year-olds or even younger are asked to justify their conclusions when, for example, Modus Tollens is the case, their justifications are not based on critical thinking nor are they based on an idea of logical necessity and creativity. More precisely, their justifications for their apparently correct conclusion are based on what is called a matching bias process(i.e., if it is true that If Mary is at school John is also at school, then it is also true that if John is not at school, Mary is not at school either. To be based on an idea of logical necessity and guided by critical thinking, children would have to say that if John not were at school and Mary was there, then it would be necessarily false to say that if Mary is at school John is also at school. It is because of such matching bias that children and even some adolescents and adult say that, in Denial of Antecedent problems, for example, if Mary is not a school, John is not at school either, which amounts to commit a fallacy or draw an incorrect conclusion. 

All that said, critical thinking and creativity are major abilities in either theoretical (e.g., to solve conditional reasoning problems) or practical domains (e.g., to critically evaluate our moral decisions and choices in real-real moral dilemmas, such as to steal/not to steal  to save a human life). To ask individuals to think aloud, or justify their answers when they are confronted with issues having to do with critical thinking and creativity  are but two procedures (not processes) to get access to their  actual critical thinking and creativity.Given that in semi-structured interviews, individuals are asked to justify their answers, which, in turn, is a thinking aloud procedure, such interviews are also a good procedure to assess individual’s critical thinking and creativity.Needless to say, to ask subjects for justifications, thinking aloud, and performing on semi-structured interviews are deeply intertwined.

I hope I has got your question and that this helps.

Best regards,

Dear members I'm looking for assesment tools for critical thinking in math.any suggestions? - ResearchGate. Available from: https://www.researchgate.net/post/Dear_members_Im_looking_for_assesment_tools_for_critical_thinking_in_mathany_suggestions [accessed Feb 2, 2017].

Bright ideas up there. I suggest multiple question may lead to ambiguity. For these critical and creative ideas you better frame thoughtful subjective questions based upon the outcomes of the course. I hope it helps!  

Is it possible to measure 'creative ability' through Multiple Choice Questions (MCQ)?

It is not viable to measure "creative ability" through MCQ.  Generally MCQ is suitable for clear cut Questions & Answers (Q&A).  Subjective Q&A format should be adopted to evaluate "creative ability" in answering a subjective question especially those questions that attract very diverse answers & justifications for the answers provided.

Abstract questions with reasoning options force individuals to think of more than the subjective mode. In subjective mode, student or individual present his or her views where as in multiple choice paper, we can extract what we are actually looking into students. Here, it is necessary that MCQ paper is made with through research objectives.

Otherwise, subjective mode is easiest way to analyze creative ability.

in fact, I suggest that figuring out the main outcome you want to achieve. what practical performance you want to see through the concept of "creative thinking".  if your definition of "creative thinking" is to find out the ability of choosing through question, then your way is fine. Method itself is not right or wrong only if it fit your research.

Yes you can. To find or choese the best solution, one needs to use some creative processes. A person may need to develop new strategies or new methods to find the sigle correct answer.  

I think yes. It depends on the type of question and the answers suggested.

Open ended questions. However look at Outliers, Malcolm Gladwell, Blanket test for creative responses. I have tried this in my Visual Art and Design Media Classes. AP and IB, and the responses gathered are right on target with cretivity.

Well -- responses seem divided on this question but may I suggest that ETS and other major testing organizations do indeed find reliable and valid ways to test for creative thinking through well designed multiple choice and some open ended questions.  The SAT and GRE all have some f these items I believe -- at least they did when I took them a few decades ago and when I was working with the College board in the 90's and early 2000's..

My first reaction would be NO. Especially because creativity is a subjective subject. What would be creative is context dependent. For instance a solution 'studying hard' would be a non-creative solution in a society where this is a normal solution, but would be creative in a society where 'taking action' is a normal solution. Since a student lives in diverse societies - like the family, the pool of friends, the classroom etc. one can never be sure if an MC answer is indeed a creative route or not.

MCQ questions, in general, are of testing recall ability (rote memorization), especially when there is only one correct answer. However, their focus can be broadened by asking them to perform an evaluation task ( as in Bloom's taxonomy) in which learners will have to compare choices and select the most accurate (or even least accurate) as the answer. In the latter,  learners get engaged in higher-order thinking, but may not engage in creating new knowledge/ideas per se. 

I work with standardized tests, questionnaires, and multiple choice tests. I'm positive about. Yes, you can't evaluate a real performance (authentic assessment, Schuman 1992, Wiggins, 1993) through a test! Unquestionable. But you can construct items siutable for describing a simulation. I suggest you to create a conceptual experiment and then to ask respondents to choise what they wolud do in a range of possibilities. The hardest part is to build the instrument.

Kind regards

Elisa

Remember that what MCQs require an exam candidate to do is to select an answer from several - i.e. to recognize which of several options is the correct one.  They never require the candidate to produce anything. Because of this they are completely useless if you want to measure a candidate's ability to produce something - hence it is questionable whether they can actually evaluate creativity.  If your candidate is producing nothing, can s/he be said to be creating anything?

For example you can use MCQs to test a medical student's ability to recall the names of the various parts of the human heart - but I challenge anyone to devise a multiple choice test to evaluate the same student's ability to palpate the heart.

@ Elisa Truffelli: Schuman 2005, says "As to our second question: Can these professional skills be assessed? Here, we again offer a qualified yes. Certainly elements of each are being assessed, but to varying degrees and with much work left to be done. The assessment challenges are greater, but we are encouraged by the number of investigators who are rising to this challenge." promotes a multi-assessment, with MC tests hardly mentioned.

It would be great if we could, financially etc., but I fear I'm not as confident as you are.

I think yes. We have been using MCQ to assess the performance of university students since long. For that, we create scenario (vignette) and ask questions based on it.  

I think it might be possible if we can create some scenarios for them to choose from however, as always there are two caveats with self-reported scales:

1. "self-flattery" syndrome (Oller, 1982), and

1. The items might be very context sensitive 

Hossein 

I believe the Abedi-Schumacher Creativity Test attempts to do this -- using subscales from the Torrance's Test of Creative Thinking. 

We all seem to be forgetting something:  In order to evaluate somebody's 'creativity' we need basically to get them to create something and then to judge their efforts. In a multiple choice item all the options presented for the candidate to choose from have already been created by the examiner.  Hence the opportunity to judge what the test candidate may have created simply doesn't arise - it is effectively precluded in a multiple choice test.

Thank you all for taking your time to type in your thoughts and experiences. It is an illuminating experience for me to go through all the the answers and I am overwhelmed with responses. It indicates people`s interest in the subject and the beauty of this collaborative platform created by Researchgate Team. 

While I am ware of the limitations of multiple choice format, I have seen this format being useful in measuring cognitive abilities like higher order thinking skills which were thought beyond the scope of MCQs. I understand from the discussion here that it is very difficult, if not impossible, to tap into creative abilities through this restrictive and limited format, but I was trying to think 'creatively' here and see if we can get some way of tapping into creative abilities through MCQs.

I feel I have got a lot of Ideas from your answers and I will work on some of them. I will share my work here on this platform with all of you.

Thank you very much for your generosity and involvement in this question.

Warm regards,

Vijai Pandey

Dear Vijai,

response to your question is both 'Yes' as well as 'No', more of 'No' and less of 'Yes'. 'Yes', if through objective questions we can cover the varied domain of 'Creativity'. But 'No' due to the fact that phenomenon of 'Creativity' is itself subjective by nature then how it can be measured by an objective instrument.

Bottom Line:  This forum encouraged strong opinions on either side of the question being asked.  Counting the 'yes's' and no's' is only indicative of those who chose to joining this conversation.  For Vijai the challenge is and will be to carefully analyze the sources suggested and indeed to find ways to measure 'creativity' creatively -- with an open mind and clarity about how subjective and objective data can be combined to discover bigger 'truths.'

Cutting through all the Eduspeak: how can you evaluate 'creativity' by means of an evaluative technique that explicitly absolves the candidate of the need to create anything?

@ Douglas -- why would you want to do that?

@Barbara - of course you wouldn't: that's my point.  If you want to evaluate somebody's ability to do x, you need to get them to do x and then judge how well they have done x. Making them do y and then inferring that if they can do y well, they must also be able to do x is risky to say the least. In a multiple choice situation the test candidate is not being required to create anything whatsoever - hence how can it possibly test creative ability?  

@ Douglas - interesting point but I do disagree with the assumption that nothing is created in multiple choice situations.  My rationale is that this assumption ignores mental representations that lead to selecting particular answers, and that of course depends on the skill of the test developer(s) in creating items that require imagination, creative thinking, etc.  If one talked with researchers and test developers at ETS, College Board, and ACT you might find some excellent examples of what I mean.  These items are usually mixed with some open-ended items to cross-validate the multiple choice items with different populations that are stratified by variables like IQ (both intellectual and social-emotional).

At some extent, yes.

@Barbara - the mental representations that lead to the selection of particular answers in MC tests are surely constructed at some time before the test.  Hence even if you rate said construction as a form of 'creativity' (which could be a whole new debate), it is something that almost certainly does not happen during the test.  Hence the test does not require the candidate to demonstrate any creativity.  

I can give examples of constructed response items to which the same criticism applies.

I would love to see your examples, Douglas.  I aim to keep a very open mind!

Try this as a generic situation that might illustrate what I mean: an exam question might be intended to test "problem-solving" and aims to do so by requiring the candidate to construct a simple algorithmic procedure to fit the specifics of the question.  This would be based on some piece of theory, the "understanding" of which would be a prerequisite to the construction of the necessary algorithm. The candidate might then be required to use the algorithm to generate a numerical answer. (In other words, a fairly typical physics question.)  

This question would be testing 'problem-solving' provided the candidate was in fact constructing the algorithm during the examination, and had not memorized a pre-prepared algorithm specifically for the exam.

However, in the course of delivering the curricular material - perhaps in an attempt to be thorough - the instructor has explicitly taught the algorithm to the students. The students have also picked up from past papers that questions of this type are to be expected, and have therefore paid close attention when this particular algorithm - and perhaps others - was taught.  In the examination they can then easily generate the demanded answer by 'running' the algorithm - which they can do with impressive fluency because because of its great familiarity through their preparation.

Have they proved that they can solve the problem - i.e. construct an algorithm themselves - or merely that they can demonstrate procedural fluency? Have they demonstrated that they 'understood' the underlying theory?

Some might have - but I think the great majority have merely shown that they can perform very fluently a well-practiced routine operation.  In addition, it was the instructor's algorithm they used - not one of their own devising.

A parallel situation: think of a music student, who shows in an examination, the ability to perform a prepared piece of music - say perhaps a Chopin nocturne - with great fluency and perhaps even great beauty. From this (and this alone) can we conclude anything about that student's ability to compose - or to improvise?  Would we not have to get the student to compose something and then evaluate the composition on its merits?  Or maybe listen to the student actually improvising?

I think we need to be very cautious of questions that require the student to do x, from which we infer that they must also have shown an ability to do p, q and r.  I think we infer too much too often in this respect -   p, q and r can often be bypassed in the demonstration of x.

I agree with the above examples and conclusions given, Douglas.  However, I would be most interested in similar examples of creativity?  I stated previously that no single method of assessing cognitive skills or competencies can reliably and validly capture whether the individual actually has the ability to be creative (although that is a questionable assumption in my vie as all people have ways to be creative, hence the ability, but may demonstrate it in ways hard to observe, measure, or evaluate.

If we want to get as close to the truth of any ability we assess, multiple measures are needed as well as caution in interpretations.

Thanks Barbara, I just noticed myself that the formulation is about measuring creative ability which is something else than actually being creative.

Thanks Douglas for showing the shortcomings of any examination in answering questions about someone's ability.

To add to the above examples consider the competence 'able to find common ground in negotiations'. Let us for argument sake use the following setup: the student is given a instruction on negotiation techniques, practices with actors and the final examination is several (at least three) cases in which the student must negotiate with an actor. In this case we can perfectly test whether the student can find common ground in negotiations in this simulation.

The problem is that the student knows it is a simulated task. The stakes are not the same as a real business deal that has to be negotiated. The preparations are different and it is not an actor opposite. The examination is at best predictive.

Now change simulation with actors in this example with multiple choice tests.

Now change 'able to find common ground in negotiations' - which is rather specific compared to 'ability to negotiate' - with creative ability.

I rest my case.

We are now perilously close to needing to explore definitional niceties.

For example: would the construction of  an original algorithm - at least one not hitherto known to the constructor - not arguably be an act of creativity?  If not why not?

Also, would one be capable of being creative in the absence of creative ability? Hence would the measurement of 'creative ability' not of necessity require us to make the candidate perform an 'act of creativity' (and then evaluate it)?

No way

Hi,

It seems that multiple choice is a closed system questionnaire, that sets several possibilities for an answer that were designed by another person. Creativity demands an open system with no boundaries, for instance, searching a term on the internet by serendipity that could yield unexpected results. Multiple choice questionnaire is just the opposite process , because it sets limits and boundaries.

I think we need to formulate multiple questions based upon Bloom's Taxonomy (highest level creating which is after evaluating). Try it please.

It seems everyone is busy?  We are cleaning up lots of debris from Hurricane Irma but since I was without internet, I am just now seeing this thread.  I wonder if using something newer -- in addition to Bloom's taxonomy--might give multiple perspective a chance?

Cant answer this question but cant help but wonder that what you measure is so limited by what tools you are using for measuring. We are using entrance tests or similar tests to measure student potential when we surely are missing out on measuring many other more important competencies!. The measuring tool can only measure what it can measure.

It is difficult to measure the student's creative abilities through multiple-choice questions, except in very specific cases because it does not exceed the application level measurement according to Bloom's Taxonomy, but the creative abilities can be measured through article questions.