Many businesses, economies, etc. going down due to blind data fallacy.
What mistakes do data scientists most often do?
These are less of mistakes and more of manipulation of data and misinformation to paint Rosy pictures।many consultants use templates and generate data sitting in office।
Many times samples are not random and there are bad questionnaire fomat not eliciting valid responses।
I put on the top of my list the following:
1) Not reading about the domain they are implementing the ML algorithm and thinking the ML will magically solve the problem
2) Taking their favorite ML algorithm as a hammer and thinking everything is a nail.
3) Confirmation bias
4) Wrong encoding- Different problems require different encodings. Just because you can feed raw data does not mean you should.
5) Wrongfully assuming independence of variables
6) Wrongfully assuming data does not change with time
I am torn between the first three on which is the top one.
Imo, dear Thomas Schuermann , it is important for all of us to notice that data science follows `Gaussian Normality` as foundational principle. This basic over-interpretation leads to the detours mentioned excellently by Arturo Geigel and Dr. Rajendra K. Gupta
In this sense, much more statistical learning procedures are required to close the data-information-knowledge-wisdom (DIKW) circuit in a subject/object-specific manner.
https://www.visualcapitalist.com/here-are-15-common-data-fallacies-to-avoid/
Have a look on list of fallacies crept in data analysis as mentioned by author
I will add one more practical example।
Fables and stories were created by global consulting firms about great Indian middle class and statistics were churned out painting overly rosy picture during post WTO euphoria।
Several MNC companies started operations and burnt their fingers।
Was this mischievous , concocted or important variables were left out? Or government data and industry Association data were used which are mostly bloated। I will stop short crying foul play। Medical practitioners many times prescribe drugs and treatment knowing or doubting they wont work and ignoring serious side effects
When source of data, intentions, inter-data dependence are contaminated it will lead to fallacies।
We need to be cautious about our judge and justice on data mistakes which is more danger now than it is necessary.
FUNDAMENTALLY, "Data" is just a sample of something larger. Even the largest samples cannot contain the entirety of information on any subject. Statistics are useful, but they are neither absolutes nor a replacement for judgment and wisdom. "Believing" data implicitly is probably the biggest single error anyone makes, whether a "professional" data scientist or any other individual.
Assuming average arithmetic returns are an unbiased representation of geometric returns.
Article The Perils of Relying on Return Data When Testing Asset Pricing Models
Lack of scientific rigor (seriousness, honesty, transparency and manipulation) in the collection, treatment and presentation / dissemination of data.
I would say the most common mistakes are in the data selection process. It is so easy to introduce a bias because of limitations in the data available which will propagate thru the analysis. So I agree with Steven Cooke comment above. Implicit trust in the Analysis Methodology and the validity of the result it generates is a way to introduce big problems. I personally always have a "big picture" review of a result to be sure it makes sense before trusting it. This is particularly important when using AI tools where the steps of the analysis are so complex that we can not follow them in specific detail.
Okay, but if the researcher is not sure about the data collection, he should not publish it or correct it with a new, safer and more sensible collection.
Thank you very much for your contributions. It would be great to have a closer look at the mechanics of data and false security. Touches behavioral attributes.
Happy for input on this.
In the meantime received enough empirical data and probably more important: behavioral observations. Thank you for your views.
Not controlling for latent variables. The response between time of survival and treatment outcome in cancer is mediated by clinician skill, insurance coverage, diet availability and affordability and intelligence of the survivor. None are featured in our abstracts. We create a bkack and white picture when it is more colourful
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication