Cronbach's alpha can only be used with numerically scored responses (ie.e., close-ended questions).
Pretesting of semi-structured interviews is usually relatively informal. So, the first interview that you do is tentative, and you may well "debrief" with the participant afterwards to get their responses to how well your questions worked for them. If the first interview does indeed work well, then try it, again and if that works well, you are likely to have done enough pretesting.
I suggest you first pilot the questionnaire with some friends or your supervisor to check that it makes sense and to get a feel for the common responses generated.
For open ended (text-based) questions the two main forms of analysis are synonym counting and a full qualitative interpretative analysis (or possibly something in between).
Be aware that text responses to open questionnaire items are often quite short unless you specifically ask for meanings / interpretations. If you are carrying out an interview you can always try to probe the interviewee further if they give a short answer.
I think this might be an important thing to look for when you are carrying out your pilot.