Hi,

You indeed can apply it. The thing is that PCA ignores the categories when computing its decomposition, and the usefulness of the method depends on what your problem or goal is. I attach three papers where we use PCA to model specific categories for classification purposes. In addition, there are some other data decomposition methods that consider the categories, such as Linear Discriminant Analysis, Biased Discriminant Analysis, etc.  I also attach another paper where we study some of these methods for email filtering.

Best

Article PCA document reconstruction for email classification

Article Highly discriminative statistical features for email classification

Article Minimizer of the Reconstruction Error for multi-class docume...

Conference Paper Document Categorization Based on Minimum Loss of Reconstruct...

You can also use Categorical Principal Component Analysis (CATPCA). Please follow the link below. 

http://www.unt.edu/rss/class/Jon/SPSS_SC/Module9/M9_CATPCA/SPSS_M9_CATPCA.htm

If data is  Categorical, than better to use rough set analysis.

Can you use PLS-DA (Partial Least Square Discriminant Analysis)? What type of data do you have?

I am using KDD dataset. So it is Mixed data( Categorical and Numerical).

Is binarizing the categorical attributes a good solution to this?

Have you checked this? It might be useful: http://www.sciencedirect.com/science/article/pii/S1877050915020190

When the variables are categorical I use Correspondence Analysis whit very good results, . It is implemented in R and many other packages

I believe that interval and ratio scale variables are prerequisite for using PCA.

Dual scaling of multiple-choice data in the response-pattern format (Nishisato, 1980) and multiple-correspondence analysis are essentially principal component analysis of categorical data.  For the exact similarity and differences between the two can be found in the book by Nishisato (1980)" Analysis of Categorical Data: Dual Scaling and Its Applications.  University of Toronto Press."

PCA is oriented to explain variance of many variables by means of some linear combinations of them (principal components). With categorical data, multiple correspondence analysis is fine. I prefer to use R because the results are clear and easy to interpret. See FactoMineR package.

I agree w/ JO Pinilla, you can use MCA (as implemented in R, by the Factominer package) used on categorical data or quantified qualitative data. See also Le, S., Josse, J. & Husson, F. (2008). FactoMineR: An R Package for Multivariate Analysis. Journal of Statistical Software, 25(1), 1–18.