Probabilistic approaches focus only on the data which is mined, to be classified, clustered, etc. No external knowledge is incorporated.

Knowledge-based approaches consult external knowledge bases such as an ontology, WordNet, etc.

Statistics or Probabilistic approach focus on understanding data in terms of mathematical models,  interpretability, hypothesis testing. Stat does not cover the learning. It will give you the detail picture of the data and you can make mathematical model. It is very much suitable for data analytic not data mining.

Machine Learning greater focus on prediction analysis of learning algorithms. In most of the machine learning algorithm there is no need of external knowledge. Its learning by observation. The algorithm may use probability or other statistics of the data for learning.  ML mostly used for building a prediction model through supervised or unsupervised learning from data and later that model can be used to predict outcome for unknown case / record. The model can be rule based (Association Rule), tree based (decision tree) or mathematical (neural network). ML is useful for both data mining and data analytic, but mostly used for data mining problem.   

Finally all are problem dependent !! 

you need be clear on what is you goal and select the right technique. 

Ok I’ve tried to take a different approach to answering the question. It may be more/less helpful to talk in terms of an example.

Imagine you’re building a system to diagnose some medical conditions. If you were to adopt a knowledge-based approach you might go and speak to some doctors who might give you a bunch of questions they would ask or things they would check. Based on the answers to the questions they can tell you that the symptoms are for condition X. For example, it might be several questions relating to a flu symptoms, and based on the answers to the questions it might determine that you have a common cold, flu, or Ebola (e.g. if you just came back from an Ebola country). Key to this approach is that you solicit the knowledge of somebody and try to encapsulate it into a system.

The probabilistic approach would take a pile of data which would contains records about symptoms for lots of people, and whether or not they had the common cold, flu or Ebola. The data-mining aspect would try to identify which features (symptoms) were relevant to which ‘class’ of (flu, etc.). This is where different techniques would come in. We might identify that we Ebola, there’s really only 2 or 3 symptoms we need to check to conclude that you have Ebola, whereas distinguishing between the common cold and normal Flu may take more. Add to this that we might have uncertainty about whether or not each symptom has actually been observed (perhaps one person’s ‘bad cough’ is another person’s ‘slight cough’). Or we might not be able to directly measure the symptom (e.g. temperature but we don’t have a thermometer). Probabilistic techniques allow us to reason about all of the symptoms, including the uncertainty we have about whether they are actually there. This is (in general) not the case with a knowledge based system.

Hope that helps

The result of probabilistic (or most statistical ) approached, is probability, but the result of knowledge based is mostly deterministic. Therefore, in some probabilistic approached, you( or machine) will decide using some thresholds that could be learned, but in knowledge based, you (or machine) use some rules that are defined based on the external knowledge. 

From the application perspective, each approach has its own application. and because for some application, it is hard to find the knowledge based techniques, the probabilistic approach is used and popular. 

I hope that helps. 

Mostafa

The basic difference is probabilistic approach doesn't require supplementary data for the data properties and distribution while in knowledge based we required some supplementary data or we can say learning data for understanding the data's properties and distribution.

thank you guys for the answer, It give me some clues. However, if i'm going to perform the analysis on the missing data problem, and looking to propose an  enhancement for the solution, Some prior solutions seemingly used the probabilistic approaches if they applied clustering and tree methods in machine learning and data mining, am i right? And, how i'm going to applied in the knowledge-based approaches? 

Yes Aisyah, If you are working on missing data problem then you can opt for both of the approaches. If you are reconstructing your missing data using the neighbor data in respect to time or space then it is probabilistic approach. Furthermore if you are using some other data set which can give you the information about the missing data. 

thank you ankur, i'm looking the solution to applied in the knowledge-based approached, how could i do that..

What you want to do actually and what data you are using? 

Thank you ankur, i'm studying on the data quality specifically in completeness dimension (missing data). I read this article:

"Assessment of data quality in accounting data with association rules"

The author elaborated the prior research of data quality in several domain approaches: probabilitstic, knowledge based and other.

From explaination given above, i could say that, the solution which applied in clustering,support vector machine, decision tree (i'm not sure)  etc could fall under the probabilistic approaches. 

However, i;'m thinking on any posibility on how to apply the knowledge base approach to do the analysis in missing data issue.

Yes you are right these all may fall under probabilistic approach but in case of  SVM and decision tree they may fall under knowledge base also, depending upon what information we are using for learning these networks.

Lets take an example to understand it better Aisyah. Suppose we have timeseries satellite images of your area and the detector is deteriorated after some time that resulted into some missing data into that image. Now to reconstruct that data there are two ways either to consider the spatial relationship of missing data pixels with neighborhood pixels or the data which is captured by the detector in past days of the same area. If we are considering spatial neighborhood relationship that means we are assuming some assumptions and trying to estimate those values but we dont have any information or knowledge about those pixels so that will fall under probabilistic approach. If we will use data of the same area captured by detector in the past that means we have some knowledge about the values in that area and their variation so it will fall under knowledge base. 

thank you very much ankur dixit..now i undestand

thank you Wei chen