Data Quality impacts the accuracy and meanings of machine learning for industrial applications. Many industry practitioners do not know how to find/use the right data. There are two levels of data: visible vs. invisible. Most the visible data are from problem areas or based on our experiences. General questions for the visible data are: First, how to find the useful data? Second, how to evaluate which data is usable? Third, which data is most critical? As for the invisible data, vert often people use either an ad-hoc or trial-and-error approach to find and seek but often the work can not be reproduced by others. We need a systematic approach to address the data quality issues in AI for industrial applications. We welcome people to share their research experiences and thoughts.
Dear Jay Lee,
I am agree with your concerns. To me, most attention is needed for appropriate methods in data cleaning and understanding the tractability issues of models for the given data at hand. A real need for data processing is model based learning which need real expertise in mathematics, statistics, physics and computing tools. However., in practice most of the data processing people lacks knowledge in mathematics, statistics and physics and have their attention on computer coding for AI without real understanding of model based learning building blocks components. Consequently, the productivity of such efforts to industry remains futile. The current flooding towards deep learning is one such example, where people think it as remedy for every thing, via a tiny thing, while ignoring the big realities in questions.
Dear and respected Jay Lee,
Collecting and filtering of huge amounts of data in a distributed system is a pretty challenging task I faced.
In the early stages, our team trained a basic ML model that had low accuracy, and the reason was - bad data. So, the most crucial part is EDA or Exploratory data analysis to investigate trends and get some knowledge from the data. Our team discovered which fields are more applicable than others and how we can use them to increase accuracy.
Also, it turned out that these factors were not unevenly distributed, for instance, Factor 1 has 99% presence in the data, whereas Factor 2 has only 1% of it, and the task was to predict Factor 2. So, we collected the data appropriately again, iterate through all the ML processes and managed to get much better accuracy (more than 90%).
Consequently, a dataset for machine learning / AI/data science ought to be carefully selected and processed. Its collection is more engineering than a research task but still has a lot of importance.
I do not think anyone would dispute the impact poor quality data has on industrial machine learning models. However, I think the bottom up data collection, cleaning and processing approaches of recent years will become redundant as digital platforms and data sharing ecosystems become more common. One of the fundamental anti-patterns in modern industrial AI is the engineering of duplicate point-to-point data pipelines.
I agree that pushing AI to the edge is becoming more important, and with that new data security models will have to emerge. In particular, there will be a need to move from centralised access control to democratised usage control - the latter will focus on how the edge nodes can utilise data when disconnected from traditional centralised security models.
I would like to share the following paper because it is a good example for the influence of data quality on AI in the field of image recognition. The Layer-wise Relevance Propagation (LRP) reveals the pixels which are being used for classification. In the paper it is shown that "flaws" in the PASCAL VOC 2007 data set lead to wrong interpretation. The classification uses e.g. a copyright text to identify pictures of horses, since in the provided data set all horse pictures have a copyright text.
Conference Paper Analyzing Classifiers: Fisher Vectors and Deep Neural Networks
To address the Data Quality, it should be considered in its totality irrespective of its usage, Once the data is complete it will certainly the serve the purpose no matter if it is for AI, ML or any other application.
To achieve the highest level of Data quality we should try to cover one or all the following dimension of data.
1. Uniqueness
2. Completeness
3. Timeliness
4. Consistency
5. Accuracy
6. Validity
The successful implementation of these dimensions in overall data quality process will decide the ultimate shape of your data.
- Another key point to note is that Data quality is not really a IT challenge to solve and there should be equal participation of Actual Data consumers or Data Creators who are responsible for it. Without complete understanding of data it would be impractical to decide if data is right or wrong.
To implement the Data quality , One should not try to "boil the ocean" and better go with iterative approach.
Step 1: Run Data Profiling to discover the high level data issues.
Step 2: Select the most Critical Data elements(CDE) those are candidate for data quality.
Step 3: Decide the DQ dimension that make sense for given CDE.
Step 4: Define and Develop Data quality rules to check the data error.
Step 5: Produce the exception report(with records details not satisfying the rule criteria) and share it with Data users or Owner
Step 6: Develop process to correct the data issues and monitor the progress.
Step 7: Repeat the Step1 to Step 6 until all the candidate data is covered for data quality process.
- Badges
- Science topic