A questionnaire can be considered reliable if it consistently produces similar results under the same conditions. In other words, reliability refers to the consistency and stability of the responses over time, across items, and among different researchers or respondents.

As Dipak Basumatari mentions there are many important aspects to consider:

Svensson, E. Application of a rank-invariant method to evaluate reliability of ordered categorical assessments. Journal of Epidemiology and Biostatistics, 1998; 3 (4):403-409

Allvin R, Ehnfors M, Rawal N, Svensson E, Idvall E. Development of a questionnaire to measure patient-reported postoperative recovery: content validity and intra-patient reliability. Journal of Evaluation in Clinical Practice 2009;15:411-19. Doi.10.1111/J.1365-2753-2008-01027-X

Svensson E, Schillberg B, Kling AM, Nyström B. Reliability of the Balanced Inventory for Spinal Disorders, a questionnaire for evaluation of outcomes in patients with various spinal disorders. Journal of Spinal Disorders & Techniques 2012; 25:196-204. doi:10.1097/BSD.0b013e31821534da.

Jonsson I, Gummeson L, Conner M, Svensson E. Assessing food choice: reliability and predictive validity of a method using food photographs in stacking boxes. Appetite 1998;30:25-37.

Gosman-Hedström, G., Svensson E. Parallel reliability of the Functional Independence Measure and the Barthel ADL index. Disability and Rehabilitation 2000; 22 (16), 702-15.

Rödholm M, Starmark J-E, Svensson E, von Essen C. Astheno-emotional disorder after aneurysmal SAH: reliability, symptomatology and relation to outcome. Acta Neurologica Scand 2001;103: 379-85.

In the course of your research - I assume to develop a better evaluation - you should be looking at many existing surveys as well. Besides your own comparisons or how you intend to improve on the existing questionnaires, you can also look at how THEY established reliability for the systems that they developed.

I agree with Steven Cooke

First and foremost, I will suggest that your task should not be done. Consider Dr. Deming's works on elimination of performance appraisal. How does classifying an employee as competent or not competent HELP THE EMPLOYEE? Or for that matter, HELP THE COMPANY? Not if its systems are broken. Classificatiion of oranges may help the quality of orange juice, but not employees. Consider the writings of Alfie Kohn amongst others. Do you grade your children as competent or not competent? See link https://deming.org/deming-on-management-performance-appraisal/

But if you do want to understand questionairres, you have to keep in mind they are opinions based upon emotions. For example, we had a safety survey and asked the workers "I know what to do in an emergency". It was always Strongly Agree. So we changed the question to "I am confident my coworkers know what to do in an emergency". That came in much more realistically. Also, there are survey methedologies to convert expert opinions expressed as categories to numerical scales (NOT LIKERT). Here is one analysis technique I used https://elsmar.com/Cove_Members/Steve's%20stuff/Survey_Analysis.pdf

Steven Prevette makes an excellent point and offers an alternative scaling system. However, to Steve's answer, I would remind him that unless the student can change the topic or focus of the thesis, a PhD STUDENT is too often required to fit the Principal's objectives. The result of the indicated survey may be less than helpful, but the exercise of preparing such a questionnaire will be the necessary learning experience in any case. Maybe after he obtains his degree he can do some useful research!

Steven Cooke - Agreed. Now I would say as someone doing research they may want to allude to - what is the purpose of the questionnaire - promotions? hiring? Pay? firing? to help the employee? Documenting the purpose may help in development of the survey. And who will take the survey? Just the manager of the employee? A "360" look around of co-workers? A review team? They employee themselves? (I wrote a lot of my own performance appraisals over the years).

I'll admit the stated objective by the US Navy in my Masters Thesis was to justify the termination of a certain weapons system. Come to find out I figured out from the exercise data with the weapon that there were certain conditions (controllable when the weapon was launched) where it worked quite well and those conditions made sense. It was certainly not the result the Navy was expecting, and I went through quite a few interesting briefings with officers much senior to me answering their questions indluding at the Pentagon. In the end, I ended up with the thesis results (about 1/2 page as compared to the 100 page thesis) being included in NWP-75, a tactical employment manual and the weapon stayed in service.

Thank you all for the helpful advice!

Steven Prevette Yes, SO much useful research depends so much on the "authority" (as in most things). Creative, flexible, and genuinely curious people also promote those pursuits in their mentees, while requiring rigorous methodology and evidence in the process to train future researchers.

Like school grades, the ubiquity of performance reviews and the like is because it takes more WORK to train and assess people more constructively! And, besides being 'taught' that way, many folks in authority don't want to put the effort into it as long as their own position is secure.

I'm not a researcher but a Lean Six Sigma Master Black Belt who has worked in healthcare for >30 years. Right now I'm working for a very large primary and specialty care medical group. In my work we would use an attribute Gage R & R to test the respondents ability to provide the same answer to the same question (multiple trails with different situations for which the answer is the same) and the ability of different respondents to do the same between each other. I've used this in a several situations because most of the data in healthcare is attribute (characteristics for which the presence is one or the other, yes/no, red/green, etc.). I've also used it when there are more than two alternatives but the "data" is still subjective like histology slide staining: Not Acceptable Light, Acceptable, Not Acceptable Dark, among a group of pathologists. Once they demonstrated the ability to agree within and between themselves, I could help the histology techs determine the stain concentration and time in the stain to produce a higher percentage of Acceptable slides. It was tedious but it worked.

Another source that might inform your task is, read up on nursing competency. This process is a long standing practice in healthcare, especially nursing. Just Google it and you'll find a multitude of opinions and studies on the subject. Good luck.