Data preprocessing is highly dependent on the type of data that you are working with.  If you are doing text categorization, then individuals have used the BOW library written by Andrew McCallum that is available at Statlib at Carnegie Mellon University.

Many record linkage methods use variants of Naïve Bayes.  To preprocess free-form names and addresses, you need name and address standardization software.  You may need other preprocessing software.

Hello Zaid Sir..It means I should go for Discretization method?

 You may check this paper

http://dl.acm.org/citation.cfm?id=1016922

Yosuf.  Excellent reference.

In the following, I was able to modify some of our preprocessing/standardization methods/software for the preprocessing.  I was also able to modify some of our frequency-based methods for record linkage that are like TF-IDF but I did not use TF-IDF.  The theoretical model extends the Naïve Bayes ideas of Nigam et al. (Machine Learning 2000) to interaction models.  I extend boosting to improving the parameters in a model to simultaneously improving the model and the parameters.  I find the 'best' 80,000 pairs and triples of words out of 120 million.  Individuals at Bell Labs and, later, at Microsoft Research had looked at all pairs and triples of words and concluded that too much noise was added when all pairs or all triples were used in the modeling.

Winkler, W. E. (2000a), “Machine Learning, Information Retrieval, and Record Linkage,” American Statistical Association, Proceedings of the Section on Survey Research Methods, 20-29. (http://www.amstat.org/sections/SRMS/Proceedings/papers/2000_003.pdf,   also available at http://www.niss.org/sites/default/files/winkler.pdf ).

Very interesting question and answers. I always thought that preprocessing includes tokenizing, lemmatizing and parsing, then applying algorithms such as Zipf law to the corpus, I mean that's what I've learned during the NLP courses. William would you mind explaining the noise in the data? Thanks.

In text categorization, you may have 10,000 or 100,000 words or more in the vocabulary that can represent dimensions in the vector-space model.  For many of the simpler classification situations, the first 1,000 words or less based on gini indexes (or whatever methods you choose for getting the vocabulary) can give most of the power in the terms of classification accuracy.  If you add in a very large number of additional words, then your classification accuracy may actually decrease slightly. 

If you think of the words as different directions in the vector-space model, then the additional vocabulary can be thought of as a type of noise because the distinguishing power of the newer words has already been taken care of in the model by the other vocabulary.

If you think of directions in R^n, then certain directions may be approximately linearly dependent on combinations of directions that have already been used.  This analogy gives some insight but is not entirely appropriate in terms of the loglinear-models situation in Nigam et al. (ML 2000) or in my paper.

"tokenizing, lemmatizing and parsing"

Dr. John H. Smith, M.D. 123 East Maple Avenue, S.W.

J. Harry Smoth 123 E. Mapel Ave, Southwest

You need to deal with typographical error and must do additional structuring of the inputs to the modeling/classification software.

In the US Decennial Census, we have no training data and must obtain 'optimal' parameters in ~500 different regions in which we must do the matching of 10^17 pairs (300 million x 300 million) in less than six weeks.

It mainly depends on the nature of the data under investigation and the purpose of the study. For example, if one uses it in classification, then the features that one has defined could be taken as a guidance to pre-process the data in a way that the training process could be efficiently performed. In some cases you may even need  to perform some smoothing activities on data, for example, by adding classes that you are aware of but might not occur in the training data.  In fact, if this smoothing does not happen then the classifier might not provide any accurate result at all.

The biggest assumption naive bayes has it's features or attributes are independent.  If we can select features in a way that they are relatively independent , they can give good results. We applied the same for text classification , you may want to take a look

http://www.hindawi.com/journals/isrn/2014/717092/abs/

Yes Hossein Hassani told about the data is right.It's depend on the nature of data which kind of technique use for which kind of data and it's behaviour about the data.

The type and quality of data involved is important.

You may use the several data preprocessing techniques available in WEKA software.

You may use  Supervised or Unsupervised methods depending on if you have class labels or do not have them.  Use this preprocessed data for the said task.

And check the classification accuracy to identify which one is the best pre-processing technique.