I think you can test the reliability for this case the same way you would do with other question formats. You can assign scores for correct Yes/No answers as well as for multiple choice questions and than proceed to correlate participants' scores in split-half/retest/parallel test designs.