I don't really understand your problem. For each language, you have two numbers, hence two percentages. What do you want do do with your sample ? Do you have other variables in mind (age, gender, previous training, ...) for which you want to see the impact ? If it is simply that you want to summarize your sample, you can look at the distribution of acceptance (from 0 to 8), and the distribution of understanding (from 0 to number of acceptances, or in percentage), or better show a 3-D plot with both variables. With the additional 2 other languages, it becomes more difficult because it would be a 4-D plot. Of course, you may have the 3 languages, say English, Spanish, Portuguese, or 3 languages ordered by preference, and then a participant may select Spanish as first language, etc.

Do you mean precision or accuracy? You use both words (that have a very different meaning) interchageably as it seems. Could you clarify this?

Further: what means "accepting a senstence"? It that a binary decision (yes or no) or does it leave the possibility that "not accepting" means that the participant does not know or does not make a decision on that sentence?

Dear Guy and Jochen,

Thank you for trying to understand my question :)

What I'm trying to calculate is this : the participants read 24 sentences which can be meaningful or not. In language 1, they are meaningful. This means that a participant having replied "yes" after reading all 8 sentences in language 1, has found 100% of the sentences meaningful. She then has to select an interpretation of the sentence which can be correct or not, and this allows me to better understand if she actually understood the sentences (when they reply "no", they simply pass on to the next sentence). Languages 2 and 3 appear meaningful, but are not. This means that if they reply "yes", they have made a certain linguistic error that I also investigate in this study. Here is a sample of the numbers I have:

Participant 1 6/8 5/6

Participant 2 5/8 4/5

=> Participant 1 has replied "yes" 6 times out of 8 in language x (75%), but only has 5 correct interpretations out of these 6 sentences (83%??).

=> Participant 2 has an accuracy of 5/8 in language x, and correctly interprets 4 out of these 5 sentences.

As you can see, my total changes in these two columns. Column 1 I can manage, it's column 2 which is more difficult but still very interesting. I struggle to see how I can combine the two columns ; they make a certain linguistic error in language 2, but still manage to select a certain interpretation.

Jochen, thank you for commenting on the difference between these two words and you are absolutely right! I guess it's accuracy seeing as you have one true value, 8.

Guy, ideally I would love to see the impact of "context" (context 1 vs context 2) on the sample, but the most important thing is to summarize.

I hope this makes more sense!

Does the participant know that there are three languages (and which sentence belongs to which language)? If so, does the participant know that only one of the language gives meaningful senstences?

As I understood, the participant get three "packs" of 8 sentences, each from a different language. The first task is then to decide if the given language provides meaningful sentences at all. If so, then all given sentences are meaningful, if not, none of the given sentences is meaningful. The participant knows this and considers the whole set of sentences to give a single answer (about the language). Then the test is actually about 3 languages, not about 8 sentences. If the participant knows than one of the languages must be meaningful, then there are 3 possible answers, and the result is either correct or not, so the accuracy can only by 0% or 100%.

Looking at the correct meaning of the 8 sentences within the (correctly selected) language, there are (as I understood) 8 possibilities to give either a correct or a wrong answer. The accuracy is then the number of correct answers divided by 8.

Dear Catrine Bang Nilsen, your explanations are clear but what do you want to do next? You had spoken of a mean in your initial presentation. Yes, of course, across the participants, I suppose, and for each language separately. In that case what is the problem if you take the mean of 5/6 and 4/5 for your second language. Of course, it would be more difficult if you want plots, because a bar chart will not be adequate. But you can consider the empirical distribution function which does not require an integer variable. Note that there is another way to see your data for the second language: considering that non-acceptance implies a bad comprehension. In your example, we would get 5/8 for the first participant and 4/8 for the second one.

Thank you again!

Jochen Wilhelm and Guy Mélard, this is the formula which has been suggested to me based on your replies and which could be simplified : (((((100/8)*x)/x)*y)/100)*((100/8)*x)

This means that if participant 1 in language 1 has found 5/8 sentences meaningful, but only managed to correctly interpret 1/5, the accuracy score is (((((100/8)*5)/5)*1)/100)*((100/8)*5) = 7.81.

Phew, this made my head spin! But I do now have a formula I can use and which allows me to compare what I wish to compare. Jochen, I'm interested in the error and how many participants make this error. This is why I want to quantify it between 0 and 100.

Thank you again!

Just a note: the formula can be simplified to (x/8)*(y/8)*100, what gives a much clearer picture as I think. However, I don't think that (or understand how) this should be indicative of the (overall?) accuracy.

Thank you so much for the simplified formula!

This does allow me to look at accuracy with regards to the first language. I do not look at the overall accuracy, but accuracy in language 1 and then the error I mentioned in the other two languages (I'll use the same formula but not for "accuracy" per se). The goal is to show that in languages 2 and 3, they make an error, but that this is due to a phenomenon called "negative transfer".

Thanks again for the patience :)

Catrine Bang Nilsen

perhaps, you may refer to:

Woodworth, R.S., & Schlosberg, H. (1954).

Experimental psychology, New York: Holt, Rinehart, and Winston.

as it is done in:

COGNITIVE SCIENCE 20, 75-100 (1996)

The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure

REGINA VOLLMEYER

University of Potsdam, Germany

BRUCE D.BURNS KEITH J.HOLYOAK

University of California, Los Angeles

https://onlinelibrary.wiley.com/doi/epdf/10.1207/s15516709cog2001_3

where, on page 11, a

"... structure score was computed by finding the proportion of correct specifications for each ... " case

... minus ... i.e.,

"adjusted for guessing by subtracting the proportion of all possible incorrect" cases

Thus, in your example

Participant 1 6/8 5/6

will get a SSc (structure score)

(6/8 + 5/6) - (2/8 + 1/6) = 1.16

Some ANOVA and hypotheses testing was done in the above 1996-study,

and certainly one newer related study is

https://www.researchgate.net/publication/5604355

Positive Transfer and Negative Transfer/ Antilearning of Problem-Solving Skills, Journal of Experimental Psychology General · March 2008 DOI: 10.1037/0096-3445.137.1.97 · by

Magda Osman, University of Cambridge

Sorry for this slightly long writing,

Good luck