When I use a frequency of 1, I get one set of answers, and when I change my frequency to 2, I get a different set. Which one should I use? The consistency cutoff I used is the same for both runs.
the frequency cut-off assigns originally non-remainder rows to the set of remainder rows. Whether this has an effect or not on your solution depends on the structure of the concerned row. For example, if a positive row (i.e. one that is sufficient for your outcome of interest) turns into a remainder because of your frequency cut-off, but this row differs on at most one position from another positive row whose frequency is above your cut-off, then the solution will not change. For example, consider the following illustrative truth table:
A B O n
-----------------
1 1 1 10
1 0 1 1
0 1 0 3
0 0 0 6
where "n" is the frequency. The solution will be A O. If you now set the frequency cut-off to 2, the second row will become a remainder, but this will not change the solution at all (provided you use the so-called parsimonious solution, which is the only solution type in QCA that you should use).
Now consider the following, slightly adapted truth table (only the value for O in the second row has changed):
A B O n
-----------------
1 1 1 10
1 0 0 1
0 1 0 3
0 0 0 6
The solution is AB O. If you now set the frequency cut-off to 2, the second row will become a remainder, and this will change the solution to A O. Thus, the effect of changing the second row from an originally negative row to a remainder has the same effect as changing it into a positive row that is above the frequency cut-off!
If the consequences of introducing frequency cut-offs are so unpredictable, the question that follows is: why would you want to use the frequency cut-off at all?
Some people argue that truth table rows with low numbers of cases should be treated as practically unobserved, but instead of a methodological argument based on external criteria that can be evaluated, this rather suggests a mere gut feeling of discomfort with small numbers of cases. Of course, such an argument would be inconsistent with a lot of writings in the QCA literature, according to which QCA is a method highly suitable for low to intermediate numbers of cases.
In other word, there is, so far, no good methodological argument why frequency cut-offs should be used at all, or under which circumstances they should be used. Thus, my advice: don't use a frequency cut-off unless you have very good methodological reasons to do so.
Thank you so much, Alrik. This is very helpful. I agree with your answer.
However, I have another question on QCA, not sure if you will be able to guide me.
The question is asked on the same day. Let me rephrase the question again here.
When I run my fsQCA test, I see the resulting "cases w/ > than 0.5 membership in term" which I don't think belongs here (in this case countries with high pollution). In other words, I'm running for configurations on countries with high pollution; however, I see the countries that have the raw data as low pollution coming out as members for the configurations for countries with high pollution, which puzzled me. So, I don't understand why.
Dear Kanix, "cases w/ > than 0.5 membership in term" just means that these cases have more than 0.5 membership in the antecedent of a truth table row, such as A*B above. Once at least a single case in your data fulfills this criterion, the row turns from a remainder to a row that will receive a "0" (antecedent not sufficient for outcome) or "1" (antecedent sufficient for outcome").
Whether this case also has a membership of more than 0.5 in the outcome you're interested in does not matter for fsQCA. If the only case that instantiates the antecedent has higher membership in the negation of the outcome, but enough other cases that are not proper members of the antecedent have a higher membership in the outcome than in the antecedent, the row will be coded positive.
Some people have long found this quite odd (see, e.g., Cooper and Glaesser 2011). In fact, with fsQCA, many such situations can occur. For example, once fsQCA has found a solution, it may be the case that the very negation of this solution is also sufficient for the outcome; or that the solution fsQCA has found is also sufficient for the negation of the outcome (see the attached file from my old book Thiem and Dusa 2013).
In essence, then, you need to be extremely careful when using fsQCA. In any case, analyses with multi-value variables (multi-value Qualitative Comparative Analysis [mvQCA], multi-value Coincidence Analysis [mvCNA], Combinational Regularity Analysis [CORA]) are much more powerful than fsQCA. While fsQCA's solution remain on the same binary presence/absence-level as csQCA's, the use of multi-value methods allows much richer inferences.
All the best
Alrik
References
Cooper, Barry, and Judith Glaesser. 2011. "Paradoxes and Pitfalls in Using Fuzzy Set QCA: Illustrations from a Critical Review of a Study of Educational Inequality." Sociological Research Online 16 (3):8.
Thiem, Alrik, and Adrian Duşa. 2013. Qualitative Comparative Analysis with R: A User's Guide. New York: Springer.