Krippendorff's alpha is essentially, a reliability index. For any type of reliability, the most important thing is that the data include the full range of talent (range of performance) of the population for which the measure in intended. While this does not outweigh the sample size itself, it means that the sample that is used needs to meet an additional specification.