Hi everyone! I'm designing an acceptability judgment task (AJT) - Likert Scale - and I'd like to keep the number of filler items as low as possible, as the number of experimental sentences is already high (3 x 2 design, 8 sentences per condition). Therefore, I wanted to ask what if there was any consensus about the minimum acceptable number of fillers in an AJT paradigm (and could you provide a reference?).
In addition, I've read that a minimum of 3 sentences per experimental condition is reccomended in AJT, while 8-12 sentences per condition is required for self-paced reading. In your opinion, what would be the ideal number of items in an AJT task (and also, could you provide a reference, if possible?).
Thank you so much for your kind attention.
As someone who's used AJT's in the past, I'd say that the number of filler items required depends very much on the saliency of what you intend to investigate. If, say, you want to study the acceptability of multiple WH-words in one utterance (e.g. who did what to whom?), this is very salient. As soon as participants become aware that this is the focus of your study, they may pay more attention to this particular construction than they'd normally do. So in order to elicit 'natural' AJ's, you would need a substantial set of filler sentences. If, say, you want to investigate the acceptability of non-subjects in sentence initial position, these may not be particularly striking at all, and you hardly need to use any filler sentences at all.
A separate issue is the number of sentences per condition. Personally, I think 3 as a minimum per condition is way too low - unless you expect all items in a condition to elicit near-identical scores because (a) items in a condition are very homogeneous -which is hard to achieve if your stimuli are sentences- and (b) all participants show near identical responses. But then again, as I come from a cognitive/usage-based background, my default expectation is one of variation (e.g. related to frequency of words and word combinations, variation between participants etc).
Good luck!
I recommend to you this book:
Cowart, Wayne. (1997) Experimental syntax: Applying objective methods to sentence judgments. Thousand Oaks: Sage Publications.
And, in general, the balance should be 1/4 of experimental conditions and 3/4 of fillers (that you could divide and have 2/3 that are highly acceptable and 1/3 that are not as control).
If you have a lot of items, you can divide them in several lists and you'll need more participants but they will take less time to do it.
There are a lot of tools for doing acceptability judgments on the internet, so if you use something of that you will gather a lot of participants in a short time, but again, you should limit the time that they spend doing the task to 30 min max (if you are doing it on the internet).
Best!
- Badges
- Science topic
- Similar topics
- Psychology