One option that the Cochrane group supports using is the CASP Qualitative Research checklist: https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf

Another useful resource is Section 21.8 "Assessing methodological strengths and limitations of qualitative studies" from the Cochrane group: https://training.cochrane.org/handbook/current/chapter-21

Article Optimising the value of the critical appraisal skills progra...

Hello dear colleague.

See if any of these tools can be useful.

https://jbi.global/critical-appraisal-tools

01 . Customer satisfaction assessment

The ISO 9000 series of standards require companies to measure and monitor information about customers' perceptions of whether the organization has met their requirements. Customer-related information can include: surveys of customers and users, feedback about products, customer requirements and customer complaints, contract information, market demand, service provision data and competitive information, etc.

In recent years, the United States, Sweden and other countries have adopted the Customer Satisfaction Index (CSI) for evaluation, which is very effective. comprehensive index. Let customers have n demands for the product and the degree of satisfaction of each demand is qi, ( i=1, 2, ... , n), then the customer satisfaction index CSI is a function of qi.

For qi, it should be determined by the market developer through random sampling of the customer base, combined with the analysis and statistics of customer complaints and quality problems of the product collected through after-sales service. The evaluation of customer satisfaction index is a rather complex matter. Enterprises, society and state agencies can commission neutral professional organizations to evaluate the customer satisfaction index of products, services and industries as needed to guide the direction of quality improvement.

02 . Lean Production

Lean Production (LP) is a manufacturing model proposed by the Massachusetts Institute of Technology (MIT) in 1990 based on its research and summary of the Toyota Production System in Japan during the International Automotive Project. Its core is the pursuit of eliminating all "waste", including inventory, and has developed a series of specific methods around this goal, gradually forming a unique production and management system.

Lean production is a system structure, personnel organization, operation methods and market supply and demand changes, so that the production system can quickly adapt to the changing needs of users, and all useless and redundant things in the production process can be streamlined, and ultimately achieve the best results in all aspects of production, including market supply and marketing.

03 . Homogeneous design

The orthogonal experimental design has two characteristics when selecting test sites: uniformly dispersed and neatly comparable. "Evenly dispersed" makes the test points representative, "neat and comparable" facilitates the analysis of test data. In order to ensure the "neat and comparable" feature, the orthogonal design requires at least q2 trials.

To reduce the number of trials, the only way to take away the requirement of neat and comparable. The homogeneous design is a test design method that only considers the uniform distribution of test points in the test range. Uniform design is similar to orthogonal design, also through a set of well-designed table - uniform table to test design, regression analysis method to analyze the results of the test.

Each uniform design table has a code or , where U denotes a uniform design, n denotes n trials to be performed, q denotes q levels for each factor, s denotes that the table has s columns, and the upper right corner of U with and without "*" represents two different types of homogeneous table. Usually, the uniform table with "*" has better uniformity. A distinctive feature of the homogeneous design is that the number of trials decreases significantly with increasing factor levels.

04 . Alignment chart

The full name of the ranking chart is "primary and secondary factors ranking chart", also known as Pareto chart. It is a method used to determine the main factors of the various factors affecting product quality, which can be used to determine the direction of quality improvement. This is because most problems in reality are usually caused by a small number of factors.

The 80/20 principle in economics is used in the field of management, and its basic principle is to distinguish between the "critical few" and the "secondary many", which helps to catch the key factors and solve the main problems. This graphic is called a ranking diagram.

05 . Balanced Scorecard

Robert Kaplan, professor of leadership development at Harvard Business School, and David P. Norton, founder and president of the Norton Institute, developed a new approach to organizational performance management after a year-long study of 12 companies that were leaders in performance measurement. The Balanced Scorecard, published in the January/February 1992 issue of the Harvard Business Review, is a new approach to organizational performance management.

The Balanced Scorecard is a break from the traditional performance management approach that focuses only on financial indicators, believing that the traditional financial accounting model can only measure what has happened in the past. In the industrial age, management methods that focus on financial metrics are still effective, but in the information society, traditional performance management methods are not comprehensive.

Organizations must gain momentum for continuous growth by investing in customers, suppliers, employees, organizational processes, technology and innovation. Based on this understanding, the balanced scorecard approach asserts that organizations should look at their performance from four perspectives: customers, business processes, learning and growth, and finances. The objectives and assessment metrics in the balanced scorecard are derived from the organization's strategy, which translates the organization's mission and strategy into tangible objectives and measurement metrics.

06 . Tolerance Design

Tolerance Design is carried out after the system design is completed and the optimal combination of controllable factors is determined by the parameter design, when the quality level of each component (parameter) is low and the parameter fluctuation range is wide.

The purpose of the tolerance design is to determine the appropriate tolerance for each parameter on the basis of the optimal conditions determined in the parameter design phase.

The basic idea of tolerance design is as follows: according to the size of the contribution (impact) of the fluctuation of each parameter to the product quality characteristics, it is necessary to consider from the point of view of economy to give a smaller tolerance to the parameters with a greater impact (e.g., replacing components of lower quality grade with components of higher quality grade).

By doing so, on the one hand, the fluctuations in quality characteristics can be further reduced, improving the stability of the product and reducing quality losses; on the other hand, the cost of the product is increased due to the increased quality level of the components. Therefore, the tolerance design stage should consider both further reducing the quality loss that still exists in the product after the parameter design, and considering that reducing the tolerance of some components will increase the cost, and weighing the pros and cons of both to take the best decision.

In short, the most reasonable tolerance for each parameter is determined by tolerance design so that the total loss (sum of quality and cost) is optimal (minimum). We know that making the tolerances of several parameters decrease requires an increase in cost, but the resulting increase in quality and decrease in loss of functional fluctuations. Therefore, it is important to find the tolerance design solution that minimizes the total loss. The main tools used for tolerance design are the quality loss function and orthogonal polynomial regression.

Parametric design and tolerance design are complementary to each other. According to the principle of parametric design, each level of products (systems, subsystems, equipment, components, parts), especially the final product delivered to customers should minimize quality fluctuations and reduce tolerances in order to improve product quality and enhance customer satisfaction; but on the other hand, each level of products should have a strong ability to withstand the impact of various disturbances (including processing errors), that is, its subordinate parts should be allowed to have a large The tolerance range.

For the subordinate parts through the tolerance design to determine the scientific and reasonable tolerance, as the basis for manufacturing stage compliance control. However, it should be noted that the conformity control here is different from the traditional quality management conformity control in two ways.

First, the inspection process should not only record the pass or fail, but also record the specific values of quality characteristics; not only give the failure rate, but also develop scientific statistical methods to give the data of quality level according to the theory of quality loss.

Second, the use of online quality control methods adapted to robust design (such as advanced SPC methods, etc.), real-time monitoring of product quality fluctuations, feedback and adjustment of process parameters; in response to problems, continuous measures to improve process design, improve product quality, reduce the total loss of the premise of the quality characteristics closer and closer to the target value, when the conditions are available, should reduce the tolerance range.

07 . Design of Experiment (DOE)

Design of Experiments (DOE) is a mathematical theory and method to study how to develop appropriate experimental protocols for effective statistical analysis of experimental data.

The design of experiments should follow three principles: randomization, local control, and replication. The purpose of randomization is to avoid bias in the experimental results due to the influence of objective and subjective systematic factors; local control is to divide the groups so that the conditions within the groups are as consistent as possible; repetition is to reduce the influence of random errors, and the purpose is still to avoid the influence of controllable systematic factors.

The experimental design can be broadly divided into four types: analytic design, group design, regression design, and homogeneous design. The analytic design is further divided into the full implementation method and partial implementation method. Analytic experimental design method is often referred to as orthogonal experimental design.

The so-called orthogonal experimental design is the use of a specification table - orthogonal table to rationalize the arrangement of experiments, the use of mathematical and statistical principles of scientific analysis of experimental results, handling multi-factorial experiments of scientific methods. The advantage of this method is that, through a small number of representative experiments, to understand the impact of each factor on the experimental index, to determine the order of factors, to find better production conditions or optimal combination of parameters.

Experience has proved that orthogonal experimental design is a fruitful method to solve multi-factor optimization problems. The orthogonal table is a table constructed on the basis of Latin square and orthogonal Latin square using combinatorial mathematical theory, which is the basic tool of orthogonal design with the characteristics of balanced dispersion and neat comparability.

The design of experiments method has more than 70 years of history, in the United States and Japan, is widely used in agriculture, pharmaceutical, chemical, mechanical, metallurgical, electronic, automotive, aviation, aerospace and almost all industrial fields, to improve product quality. The American automotive industry standard QS 9000 "Quality System Requirements" has listed design of experiments as one of the techniques that must be applied. The famous parametric design is also developed on the basis of orthogonal experimental design.

In addition, experimental design can not only find the optimal combination of parameters, but also, in many cases, set up error columns and conduct analysis of variance to qualitatively determine the impact of various error factors, such as environmental factors and processing errors, on the desired product characteristics, and take improvement measures to eliminate the impact of these errors.

Therefore, for some simple engineering problems, a satisfactory and robust design solution can be obtained by directly applying the design of experiments method. Design of experiments can also be applied to improve business management, adjust product structure, and develop production plans for more efficient production, etc.

08 . Benchmarking method

Benchmarking is a continuous process of comparing and measuring the performance, quality and after-sales service of products against the strongest competitors or companies that have become leaders in the industry, and taking improvement measures.

The benchmarking method includes two important aspects, on the one hand, the development of a plan to constantly search for and set up benchmarks of advanced domestic and international levels, and to discover the gaps in our own products through comparison and comprehensive thinking.

On the other hand, we constantly take measures to improve design, technology and quality management, to take the strengths of others and make up for their weaknesses, to continuously improve the technical and quality level of our products, to surpass all competitors and to reach and maintain the world's advanced level. Adopting the horizontal comparison method is not simply imitation, but creative borrowing.

Through in-depth thinking and research, we will gather the strengths of all schools of thought and carry out technological innovation to achieve breakthroughs in product performance. Only by mastering the breakthrough technology can we lead the world. In order to better implement the horizontal comparison method, relevant databases should be established and constantly updated. The horizontal comparison method has been widely used in the U.S. and has achieved obvious results.

09 . Statistical process control

Statistical Process Control (SPC) was proposed by Dr. Hughart in the 1920's. Since the Second World War, SPC has gradually become the basic method for online quality control in western industrial countries.

According to SPC theory, the fluctuation of product quality characteristics is the root cause of quality problems, and the quality fluctuation has statistical regularity, and the abnormality can be found through control charts, and the cause of the abnormality can be identified and eliminated through process control and diagnosis theory (SPCD).

The commonly used Hughart control charts are.

1 . Mean-Reverse (x-R) control chart

2 . Mean - standard deviation (x-S) control chart

3 . Median-polar deviation (x-R) control chart

4 . Single-value-shifted polar deviation (x-Rs) control chart

5 . Nonconformity rate (P) control chart

6 . The number of non-conforming products (Pn) control chart

7 . Defect number (C) control chart

8 . Number of defects per unit (u) control chart, etc.

The SPC method is a powerful tool to keep the production line stable and reduce quality fluctuations.

In recent years, the SPC method has gained further development, for example, Boeing introduced a new set of supplier quality assurance specifications Dl-9000 in order to implement the robust design idea, the main change is the requirement to establish an Advanced Quality System (AQS for short).

The AQS system incorporates the concept of Taguchi's quality loss into the quality management of the manufacturing stage, and proposes a set of manufacturing quality control requirements that are compatible with robust design.

The AQS system first requires the identification of key characteristics of products in the manufacturing stage, and for these key characteristics and the components they involve, process robust design is required in order to identify robust processes.

To establish monitoring measures for key characteristics in manufacturing, in addition to applying the conventional control charts of SPC, AQS gives three kinds of small batch control charts, i.e., single-value moving polar-difference control chart, target control chart and proportional control chart, two improved control charts, i.e., moving average control chart and geometric moving average control chart, in addition to some measures to improve the sensitivity of control chart monitoring.

According to the monitoring situation and the actual needs, improve the process parameters or improve the process design, correct any factors that cause quality fluctuations of the machine, material and method ring, so as to achieve continuous improvement of quality.

10 . Brainstorming method

Brainstorming method, also known as intellectual stimulation method, was proposed by Osborne, the founder of modern creation science, as a collective training method for creative ability. It organizes all the members of a group together, so that each member has no fear of expressing their own ideas, neither afraid of ridicule nor criticism and blame, is a way to enable everyone to put forward a large number of new ideas, creative problem solving the most effective.

It has four basic principles.

1 . Exclude critical criticism, comments on the proposed ideas to be made later.

2 . Encourage "free imagination". The more absurd the proposed idea, the more valuable it may be.

3 . Require a certain number of ideas to be presented. The more ideas you present, the more likely you are to get more valuable ideas.

Michel Dalmedico

When synthesizing qualitative data in systematic reviews, you are performing a Meta-synthesis. There are tools for reporting qualitative data, such as:

Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol (2012) 12:181. 10.1186/1471-2288-12-181.

Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, et al. Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual). PLoS Med (2015) 12:e1001895. 10.1371.

Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, et al. Supplemental Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions (2016). Available from: http://methods.cochrane.org/qi/supplemental-handbook-guidance