Here is my possibly naive response. Why not collect think-aloud protocols, in the tradition of Newell and Simon, and ask trained judges to score the protocols using a rubric? (A dream of mine has always been to use AI to semi-automatic protocol analysis -- but that's hard.)

Dear Masoud,

It is generally alleged that critical thinking refers to one’s capacity to think clearly and rationally about what to do or what to believe; that it includes one’s ability to engage in reflective and independent thinking; that peoples with critical thinking skills are capable of, for example, understanding the logical connections between ideas, identifying, constructing and evaluating arguments, detecting inconsistencies and common mistakes in reasoning, solving problems in a systematic way, noting the relevance and importance of ideas, and reflecting on the justification of one's own beliefs and values.

Critical thinking, therefore, is not a matter of compilation of information. A person with a good memory and who knows a lot of facts is not necessarily good at critical thinking. A critical thinker is able to deduce consequences from premises he knows, and he knows how to use information to solve problems, and to seek important sources of information to inform himself/herself.

A common misunderstanding about critical thinking is to judge it as being critical of other people. Although critical thinking skills can be used in exposing logical fallacies and bad reasoning, critical thinking can also play an important role in cooperative reasoning and constructive tasks. Critical thinking, therefore, can help us acquire knowledge, improve our theories, and strengthen arguments. Of course, we can use critical thinking to improve social institutions.

Other common misunderstanding about critical thinking is that it hinders one’s creativity because it requires following the rules of logic and rationality, and creativity might require breaking rules. Critical thinking, however, is quite consistent with thinking, say, "out-of-the-line", challenging consensus and pursuing divergent ideas and less popular approaches. If anything, critical thinking is part and parcel of creativity because we need critical thinking to evaluate and improve our creative ideas.  

The following example clearly illustrates that creativity or divergent thinking and critical thinking are highly consistent with each other. It is said that Karl F. Gauss (1977-1855) left his primary school teacher highly perplex when, at the age of 8 years, he gave a creative answer to the following problem his teacher had written shortly before on the blackboard: “ What is the sum of 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8?”. All of a sudden, and apparently without having time enough to perform the respective operation of adding, Gauss replied that the result of the arithmetical operation at hand was 36. “Why is this so?” -- The teacher asked again. She became more perplex when Gauss replied that this was so because 4 X 9 = 36. “I cannot understand” -- The teacher replied and went on:  “Why did you perform an operation of multiplication instead of an addition operation?”  “I did that -- Gauss replied -- because I easily realized that 1+ 8 = 9; “2 +7 = 9; 3 + 6 = 9; and 4 + 5 = 9. Hence, 4 X 9 = 36”. Even if it were given by a non-expert adult in mathematics, we would certainly say that Gauss’ answer and way of thinking was highly creative and was a clear example of divergent, new, and insightful thinking. As critical thinking also includes one’s ability to engage in reflective and independent thinking, Gauss’ answer is telling example of both creativity and critical thinking.

The example also substantiates a procedure of how to get access to one’s critical thinking. In other words, to ask individuals to justify their answers or performance on issues wherein critical thinking may be involved is a kind of “think aloud” procedure, a procedure often employed by researchers to get access to underlying psychological processes at issue, for example, in one’s critical thinking and one’s creative thinking or acting.   

Critical thinking often involves evaluating arguments and detecting inconsistencies and common mistakes in reasoning. Because of this, the domain of conditional reasoning is an appropriate domain for one to observe individuals’ ability to think critically, that is, to see, for example, how individuals of different ages solve each of four classical logical arguments (i.e., Modus Ponens, Modus Tollens, Denial of Antecedent, and Affirmation of Consequent). Of course, for one to know individual’s critical thinking while solving these logical arguments or problems one has to ask them to justify (or think aloud) their responses and solutions.

As you know, conditional reasoning has do to with “if…then statements”.  Modus Ponens: If p (Mary is at school), then q (John is also at school); p is the case (Mary is a school). Hence, q is necessarily the case (John is also at school). Modus Tollens: If p (Mary is at school), then q (John is also at school); not q is necessarily the case (John is not a school): Hence, not p is necessarily the case (Mary is not at school). Denial of Antecedent:  If p (Mary is at school), then q (John is also at school); not p is the case (Mary is not at school). Hence, nothing can be concluded about q (i.e., John may be or may be not at school). Affirmation of Consequent: If p (Mary is at school), then q (John is also at school; q is the case (John is at school). Hence, nothing can be concluded about p (Mary may be or may be not at school). There is mounting evidence that shows that even 5-year-olds draw the “correct” conclusion when Modus Ponens is the case (i.e., John is at school), that 9-year-olds draw the “correct” conclusion when Modus Tollens is the case (i.e., Mary it not at school either) , and that even some adolescents and adults are not capable of drawing the correct conclusion when Affirmation of Consequent and Denial of Antecedent are the case (i.e., It is not possible to say whether or not Mary/John are at school). Of course, children are not capable of dealing with either Affirmation of Consequent or Denial of Antecedent problems. It is worth mentioning, however, that when 9- year-olds or even younger are asked to justify their conclusions when, for example, Modus Tollens is the case, their justifications are not based on critical thinking nor are they based on an idea of logical necessity. More precisely, their justifications for their apparently correct conclusion are based on what is called a  "matching bias" process (i.e., If it is true that If Mary is at school John is also at school, then it is also true that if John is not at school, Mary is not at school either. To be based on an idea of logical necessity and guided by critical thinking, children would have to say that if John not were at school and Mary was there, then it would be necessarily false to say that if Mary is at school John is also at school. It is because of such "matching bias" that children and even some adolescents and adult say that, in Denial of Antecedent problems, for example, if Mary is not a school, John is not at school either, which amounts to commit a fallacy or draw an incorrect conclusion.  

All that said, critical thinking is a major ability in either theoretical (e.g., to solve conditional reasoning problems) or practical domains (e.g., to critically evaluate our moral decisions and choices in real-real moral dilemmas, such as to steal or not to steal to save a human life). To ask individuals to think aloud, or justify their answers when they are confronted with issues having to do with critical thinking are but two procedures (not processes) of getting access to their critical thinking. Given that in semi-structured interviews, individuals are asked to justify their answer, which, in turn, is a thinking aloud procedure, such interviews are also a good procedure to assess individual’s critical thinking. Needless to say, to ask subjects for justifications, thinking aloud, and performing on semi-structured interviews are deeply intertwined.

I hope I has got your question and that this helps.

Best regards.

Hi Masoud. Great question! Your question asks for valid interview questions to derive the presence of critical thinking skills, and I think the response from Orlando Lourenço is quite exhaustive in this respect. The only thing I would add to his response is to point out that all things occur within a specific context, and thus the first thing you will need to develop is a clear focus concerning which specific critical thinking skills or abilities you are trying to tease out from the interview, and what area of teaching and/or learning these are being applied to. It is always best to delineate the focus of our inquiries more narrowly, rather than more broadly, when we seek to ascertain or evaluate the presence of a particular attribute or skill in the world. Hope this is helpful, and all the best with it...