SUCCESS FOR EACH QUESTION is determined by the probability of success: P = (s + 1) / (n + 2). There are 4 choices in each question and one correct answer; therefore, P = (1 - 1) / (4 + 2) = 2/6 = 0.33. The probability of failure is Q = 1 - P or 1 - 0.33 = 0.67.

WHETHER 83% CORRECT ANSWER IS DUE TO CHANCE? The answer to this question may be determine by the DeMoivre-Laplace Theorem.

Z = (S - np) / sqrt(npq)

S = 0.83(8) = 6.64;

n = 8; p = 0.33; q = 0.67

The answer is Z = 3.01. The percentage probability is 0.9991. If we use 95% confidence interval, 0.991 > 0.95. This is NOT chance happening. To look at it in terms of pValue: 1 - F(Z) or 1 - 0.991 = 0.001.

The answer will be a little more complicated if you want to account for the fact in a meaningful way that there are multiple people or multiple questions. But that being said, a very simple approach would be pool all questions and people together, count the number of correct responses and total responses, and use a binomial test or chi-square goodness-of-fit test.

You might also want to use this approach on the four questions individually. I would prefer this approach to pooling all the questions.

The following example can be run in R, or at https://rdrr.io/snippets/ . The information after the #'s are comments or results.

Here, I used 20 people, 4 questions each, 66 correct answers, and 14 incorrect answers. I compared the correct answers against a theoretical proportion of 0.25.

People = 20

Questions = 4

Trials = People * Questions

## 80 total questions in this case

Successes = 66

## About .83 success rate

Failures = Trials - Successes

binom.test(Successes, Trials, 0.25)

### Exact binomial test

### number of successes = 66, number of trials = 80, p-value < 2.2e-16

### alternative hypothesis: true probability of success is not equal to 0.25

### 95 percent confidence interval:

### 0.7238460 0.9008895

### sample estimates:

### probability of success

### 0.825

chisq.test(c(Successes, Failures), p = c(0.25, 0.75))

### Chi-squared test for given probabilities

### data: c(Successes, Failures)

### X-squared = 141.07, df = 1, p-value < 2.2e-16

Hello Harley,

The answer depends on your specific question. If what you are trying to determine is, do people pick answers at random on this 8-item MC test, then Salvatore S. Mangiafico 's response will give the exact probability (under the null hypothesis that prob(correct) = 0.25); Paul's answer uses the large-sample approximation, but to what I believe is the wrong target proportion (0.33, instead of 0.25).

A significant result means that the performance (selection of answers) was not random, but purposive (e.g., informed by knowledge). There's ample literature to demonstrate that humans do not behave in random ways, BTW.

Good luck with your work.