As far as I know, C-means is a clustering (unsupervised learning) where SVM is a classifier (supervised learning). We use C-means when we want to cluster data which their labels are unknown to us, however, SVM is firstly trained with the train data (labeled data) and then is used to classify the test data (unlabeled data). I hope you would understand the whole differences between these two types of algorithms.

Yes i could understand. Consider i have some experimentally verified data.

And i know how many groups i need to classify.. In any way can i use fuzzy c mean?

Surely. You can group (cluster) your data automatically using C-means. You may have any number of different clusters you want (where SVM is a binary classifier and data should be grouped in exactly two classes). Additionally, since C-means is a Fuzzy or soft clustering algorithm (opposite of Hard clustering), each data element can belong to more than one cluster and its membership level in each cluster is a fuzzy value between zero and one. I know you are familiar with fuzzy sets and systems.

Another advantages of fuzzy c-means is that you can use this method to produce fuzzy roles to run as a fuzzy classifier. In contrast, SVM is a black box (lack of transparency). It means that SVM is a powerful tool for data classification but its behavior is somehow unclear and vague.

Thanks Borhan. Let i say my problem and kindly say me can fuzzy c mean approach can be adopted.

System contains N items, classified into M subset (item location); Subset contains items with same location; The problem is, for a given unknown item query , how can we identify the subset to which it belongs to (location). Once again thanks in advance..

Firstly, I have to write about some specific words in Pattern Recognition literature. You have two types of items, those you know their locations and those who you do not. In literature, we call the first group as Train data, which we use them to classify the other group. The second group of items is entitled as Test data, because we have no idea about their location. As you mentioned, each data is located in one specific place (which may be known or unknown). We assume their location as their Class. For example, in your problem you have M classes. To sum up, your problem is:

We have N item in M classes. We want to adopt an intelligent method to learn Class labels from Train data and predict the class labels of Test data.

This kind of problem is known as Classification or Supervised Clustering. In PR field of study, we have another type of problem which is called Unsupervised Clustering (or just Clustering). In this type of problem, the class label of all items are unknown. It means we do not Train data. Therefore, we can not teach the algorithm to learn class label. For example, assume we have N item which their locations are unknown. However, we know they are can be categorized according to their locations. Consequently, we adopt a clustering algorithm to assign each item to a cluster. If the method works, the output will be M different clusters!

Now, I'm going to talk about C-Means. This algorithm is a type of clustering and do not care about the class (locations) of your items. Therefore, you can not directly use C-Means to classify your items. However, if you want to adopt this cute algorithm, you can find a solution. There are methods to generate Fuzzy Inference System structure from items using Fuzzy C-Means. If you are familiar with Matlab, you can try genfis3. With the aid of this function you can produce a Fuzzy role set and using them to classify Test items.

thanks for your explanation.. I can completely understood what you said.. but my problem is that i have the data (experimental) and i know it belongs to so on so group... but i dont have the standard rule (or parameter) that decides the data belonging to some X group .. in this case i cant go with FIS i think so.. may be i need to analyze the data and then i have to find the standard rule for each group.. any way thanks for the interest show to reply my queries... Also, any other Supervised approach can be adopted for this problem apart from NN, GA, or SVM???

I hope I could help you to solve the problem. In the case of FIS, you can utilize several FIS rule generators. As I mentioned before, you can find three automatic rule generators in Matlab Fuzzy toolbox (genfis1, genfis2 and genfis3).

Despite of that, there are several classifiers you can adopt. The simplest algorithm is Nearest Neighbor (or k-nearest neighbor). This method is very simple and common. For more information search for kNN classifier. ANFIS is another classifier which uses Fuzzy rules and Neural Networks all together. Boosting and AdaBoost are other types of classifier which are powerful. Furthermore, there are a number of more sophisticated techniques based on Bayesian or Markov networks such as GMM and HMM. To begin, take a look at http://en.wikipedia.org/wiki/Classifier_(mathematics)

Let me tell you a piece of helpful advice. Prior to be tide up with code and programming, determine your solution. Search internet and libraries to find an appropriate method (e.g., kNN) and when you feel sure, dive to code. Another useful approach, at least in my point of view, is to determine the appropriate tools. Search for Pattern Recognition or Machine Learning toolboxes in programming languages which you are familiar with. Then investigate which solutions they provide for your problem. For example, Wekka, PRSD Studio and Matlab have many predefined solutions for classification problems.

If you faced with any problem, do not hesitate to ask.

Thanks for the guidance sure will contact you in near future regarding this. :-)

How to convert my DATA into input for machine learning?

consider i have a set of strings as single input ex : ABCDEFGHIJK, like this (varied length) i have more inputs. In what way i can maintain the sequence order?

I think the main problem is to standardize your input so the classifier can read it consistently.

Then, you can suppress the classifications that are common, and accentuate those that are different across the output of the classifier. The main problem with this approach, is by suppressing the common mode, you make your classifier less sensitive to alternate or degenerate codings. If that is not a problem, for your proposed sources of data, you need not worry about it. I find it extremely important to dealing with degenerate coding techniques like are used in the brain.

never heard of ant clustering, what is that? I mean the general idea of what the algorithm does, and what it can be used for

Does SVM or GA can be used for crop classification problem in Remote Sensing ?

if so, which is better? Is there any comparison of both?

@ Aleksandar: Good job. In my point of view, methods based of Evolutionary Algorithm and Swarm Optimization can achieve considerable results in different kind of problems. Furthermore, they are very interesting and attractive.

@ Anand: As far as I know, GA is not directly used as a classifier. However, there are a lot of GA-based classifiers such as ZCS, LCS and XCS.

By the way, you can combine both GA and SVM to solve your problem. By the aid of GA you can find the best discriminant features, optimal subset of instances and the most effective SVM parameter values. On the other hand, you can classify your data by SVM. In this way you can solve the problem more efficiently.

thanks borhan!

Does GA based classifier used for crop identification using remote sensing?

anand

where do I get links for further research in SVM and GA?

Dear Anand,

To solve a SV problem you should write a code to solve quadratic optimization problem. Fortunately, there are several ready-to-use packages of SVM in many programming languages. For example, take a look at http://www.support-vector-machines.org/SVM_soft.html . You can find Java version too. However, I have used Matlab and C/C++ versions. In time complexity point of view, C/C++ versions are more interesting.

As far as I understood, your problem is multi-class classification. SVM originally is a binary classifier. However, you can combine several SVMs (one-versus-all) or use modified versions which are able to solve multi-class problems.

In case of GA, there are many diverse versions of GA algorithms and implementations. You can select your desirable/suitable GA algorithm and implement that in any programming languages you want. Or, you may find an already implemented version and just run that.

As you probably know, GA is an optimization tool. We use it to find the minimum of maximum point of one or several functions which are called Fitness Functions. have you specified your fitness function in this problem? I mean, what is the duty of GA in your approach?

In my point of view, you can use GA in several ways. Firstly, you may use GA to find most accurate, relevant and discriminant features before using SVM to classify your images to their groups. I this way, you might define your potentially suitable set of features and ask GA to find the best subset among them. You can also exploit GA as a instance/example selection algorithm to find the best subset of your data a traiing set to be used from SVM. For more information, please search and study about dimension reduction, feature/attribute selection, instance/example/data reduction, wrapper and filter.

There is another application of GA regarding your problem. You can apply GA to find the best parameter setting of SVM. in this way, any individual of GA may run an SVM and at the final generation you can find the best parameter values of SVM. Please note, in this approach, you should use a rapid implementation of SVM because GA wants to run SVM so many times.

Regards,

Borhan

It is a brilliant idea to utilize GA do the feature selection and parameter optimization for SVM, but I wonder whether GA Can aid the SVM to perform feature selection and instance reduction, as well as the parameter optimization simultaneously very effectively?