I have observed a notable trend wherein individuals from diverse fields are transitioning towards domains such as data science, data analytics, and machine learning (ML). Concurrently, there is a growing interest in exploring the synergies between these fields and ML to augment productivity. However, beyond mere application knowledge, there arises an important question: How can one effectively impart a deeper understanding of the conceptual frameworks and fundamental principles underlying these ML algorithms, while abstracting away from the technical details
The conceptual frameworks and fundamental principles underlying ML and data science (DS) without technical details can be summarized in the following way:
The stated purpose of ML and DS is to infer some insights (quantitative and qualitative) from an empirical collection of various data obtained usually without preliminary planning (just because the data are ‘naturally’ collected). This approach does not offer a causal relationship between input and output variables (or factors, features). It explores merely some empirical patterns and associations / correlations in past data using various statistical methods. Then it attempts to make a formal projection of these patterns to forecast some future data / patterns.
However, collected data (big or small) always represent the past. When data are collected without actively planned experimentation they contain little useful signal and a lot of noise and spurious correlations. At the same time, only the stable signal that can be extended into the future can provide meaningful forecasts using validated models. A pattern in the past data is usually unstable and disappears in the longer run, but it is mistaken for a stable signal. It is difficult in general to tell the signal from the noise. It requires a lot of subject matter knowledge. Data must be placed in a business context. Without it, there is no sure way to differentiate the useful signal from the useless noise. A forecast based mostly on noisy data will not be reliable, or forecasting (say, classification) will be inundated by false positives or false negatives.
The bottom line: the key to meaningful data analysis and forecasting is understanding the relationships of the variables to the forecasted outcomes, i.e. causality (similar to developing a good physical theory). The current tendency of applying just statistical algorithms to an empirical collection of data in the framework of ML and/or DS is unreliable and will fail in the long run.
There are several effective ways to ensure clarity when discussing ML algorithms with non-technical individuals:
The key is to translate the technical details into plain language, use relatable examples, and focus on the practical applications and implications rather than the inner workings of the algorithms. With the right approach, the non-technical audience can gain a clearer understanding of how ML systems work and their real-world impacts. Israel Musa Israel Musa
- Badges
- Science topic
- Similar topics
- Physical Sciences
- Materials Science
- Materials
- Inorganic Materials
- Ceramics
- Glass