How do you measure the performance/utilization of a healthcare department/clinic? What methods/approaches/techniques/tools do you use or suggest? You have to consider the human factor. The resources of healthcare include nurses, doctors, technicians, assistants, etc.
Is the performance of a doctor measured by the number of patients treated? Or is the time spent by a nurse with patients, or is the performance of a clinic/department/hospital measured by the number of patients treated during the day?
There is no only one metric that allows comparing and measuring the performance of different units. Multiple metrics are usually used for
measuring of performance (efficiency) of different units or departments/clinics.
The question is how to combine all these various metrics in one producing unit's score. I think that one of the best methods for doing this is the so-called data envelopment analysis (DEA). In short, DEA can be used to measure comparative efficiency (performance) of hospitals, physicians, group practices, or any other producing units using a so-called scoring function.
If one producer (DMU1) is better than another producer (DMU2) by either making more output with the same input or making the same output with less input then this producer (DMU1) is more efficient than another one (DMU2).
The procedure of comparing the producers aimed at finding efficient ones vs. less efficient and by how much can be formulated as a linear optimization problem.
See, for example, an introductory presentation attached (slides 1-24).
There are also quite a few publications with various advancements and details of this approach, including free software. See, for example, JOMS attached papers for the analysis of the efficiency of the primary care clinics.....
Dear Atalan,
Hello
First of all, I would like to thank you for asking this question and making this discussion!
Performance evaluation/assessment has been always one of the most interesting as well as important issues to study in different scientific/academic disciplines. In this regards, it could be interesting that this issue has been addressed prevalently in all academic studies and/or researches which are somehow related to the medical management.
As a short answer to your question, I respectfully recommend you to take the following steps;
1. Try to have an operational (not theoretic) definition for performance for your concerned medical units (departments).
2. Try to find a set of indices characterizing the notion of performance for your concerned medical departments.
3. Data gathering by standard or researcher – made questionnaires based on the performance indices defined during the previous step in order to quantitatively measure the indices.
4. Utilize the Data Envelopment Analysis (DEA) methodology to analyze and also compare the performance of unites included in the concerned medical department (or even, comparison of all different medical centers/departments in your research scope)
I hope the above could help you solve your problem!
Good luck!
Ali
Dear Atalan,
Hello
First of all, I would like to thank you for asking this question and making this discussion!
Performance evaluation/assessment has been always one of the most interesting as well as important issues to study in different scientific/academic disciplines. In this regards, it could be interesting that this issue has been addressed prevalently in all academic studies and/or researches which are somehow related to the medical management.
As a short answer to your question, I respectfully recommend you to take the following steps;
1. Try to have an operational (not theoretic) definition for performance for your concerned medical units (departments).
2. Try to find a set of indices characterizing the notion of performance for your concerned medical departments.
3. Data gathering by standard or researcher – made questionnaires based on the performance indices defined during the previous step in order to quantitatively measure the indices.
4. Utilize the Data Envelopment Analysis (DEA) methodology to analyze and also compare the performance of unites included in the concerned medical department (or even, comparison of all different medical centers/departments in your research scope)
I hope the above could help you solve your problem!
Good luck!
Ali
Dr. Alexander Kolker
Thank you for your contribution. But there are still unclear situations in my thoughts. How is the actual performance of a health resource measured? For example, a nurse's work rate was 85% based on a hospital's data, while the nurse's work rate was calculated as 47% based on simulation data (of course, simulation models employ people as robots, so this rate can be perceived as minimum, but not maximum 85%). Therefore, how can the real efficiency rate be achieved in hospitals/clinics/departments?
Dr Ali Mollajan, thank you for your contribution. But I would like to share with you the thoughts I told Dr. Alexander Kolker. There are unclear situations in my thoughts. How is the actual performance of a health resource measured? Obviously, the data held in hospitals is worrying and often inconsistent, so our work becomes difficult. Usually it provides the possibility to compare with real data using simulation programs (discrete event simulation). In my studies so far, there are obvious differences between the current state and the simulated state. For example, a nurse's work rate was 85% based on a hospital's data, while the nurse's work rate was calculated as 47% based on simulation data (of course, simulation models employ people as robots, so this rate can be perceived as minimum, but not maximum 85%). Therefore, how can the real efficiency rate be achieved in hospitals/clinics/departments?
Abdulkadir Atalan The hospital units managers take into consideration: 1. Standards for RN to Patient ratios for acute care patients. For example, an adult acute care general medical-surgical unit may operate with a 1:5 RN:Patient ratio, supported by a 1:8 Unlicensed Assistive Personnel (such as a Certified Nurse Aid):Patient ratio. 2. State government regulated nurse to patient ratios. 3. Organized labor contract mandates 4. External worked and paid hours per patient day benchmark standards for peer hospitals.
Traditionally productivity in hospital nursing units has been measured by hours per patient day (HPPD) meaning total hours are divided by total patient days. HPPD=Total nursing hours/Patient days
Productivity Benchmarking.
Productivity must be considered as a relative measure; the calculated ratio should be either compared to a similar unit, or compared to the productivity ratio of the same unit in previous years. Such comparisons characterize benchmarking. Many organizations use benchmarking to help set the direction for change.
Historical Benchmarking is monitoring an operational units’ own productivity or performance over the last few years. Another way of benchmarking is to identify the best practices (best productivity ratios of similar units) across health organizations and incorporate them in one’s own.
Retention is more likely when nurses can work to their full scope of practice and utilization levels are below 83%...Units where nurses frequently work at or beyond maximum utilization must reduce utilization and implement acceptable standards…’
O’Brien-Pallas, L., et al, 2004. ‘Evidenced-based Standards for Measuring Nurse Staffing and Performance’. Final Research Report.
Canadian Health Services Research Foundation
Dear Dr. Atalan,
Hello there,
First of all, I would like to thank you for your message as well as your kind words!
In order to prepare a proper answer to your inquiry, I require to ask you a question based on your recent message. In your message, you have mentioned that you have somehow employed a simulated study as well (that has resulted in a different finding). Regarding this matter, I would like to ask you :" what basis (what information/what distribution) have you used to conduct your simulation study?
you know that in any helpful statistical simulation study you have to know about the probability distribution of every random variable you are concerned about it. In addition, it is clear that estimation of pdf for each of your concerned variables depends on having some real information about it. That is, real information you have mentioned that you have had some trouble with it. In fact, on the basis the above, existence of the difference is natural. But, the most important thing we should notice is that the challenge of difficulty in having our required information is different from the effective approach to deal with (evaluate) the problem of performance evaluation.
Lastly, I would like to emphasize that the simulation study could be one the helpful ways to analyze. However, it is clrear that reasonable and acceptable results may be achieved from real and correct data!
Sincerely,
Ali
I am addressing to you with hope to set the scientific contacts in the area “Mathematical Models and Methods of Effective Estimation in Multi-Objective Problems of Intellectual Diagnostics of Systems under Uncertainty" (Methodology, computational methods and realization in interactive decision support software). The proposed methodology for solving the problem of diagnosing the state of patients in the medical monitoring systems based on the concepts of tolerance control, trend analysis, and statistical analysis using the methods of machine learning.
Hello dr AbdulKadir
I recommend also to have hints about healthcare lean six sigma project about CTQ ( critical to quality) converting qualitative data into quantitative data like medical error , length of stay , waiting time ... etc. And check video about attribute agreement analysis (3A) in minitab https://support.minitab.com/en-us/minitab/19/help-and-how-to/quality-and-process-improvement/measurement-system-analysis/how-to/attribute-agreement-analysis/attribute-agreement-analysis/before-you-start/example/ https://support.minitab.com/en-us/minitab/19/help-and-how-to/quality-and-process-improvement/measurement-system-analysis/how-to/attribute-agreement-analysis/attribute-agreement-analysis/before-you-start/overview/ There are a lot of papers in RG site https://www.researchgate.net/search.Search.html?type=publication&query=healthcare%20six%20sigma
Good luck