At the most fundamental point, linear methods can only solve problems that are linearly separable (usually via a hyperplane).  If you can solve it with a linear method, you're usually better off.  However, if linear isn't working for your particular problem, the next step is to use a nonlinear method, which typically involves applying some type of transformation to your input dataset.  After the transformation, many techniques then try to use a linear method for separation.  

Some potentially interesting reading: http://www.simafore.com/blog/bid/113227/How-support-vector-machines-use-kernel-functions-to-classify-data

Also, one of the cornerstone books of Statistical Learning (another phrase for Machine Learning) is available for free: http://www-bcf.usc.edu/~gareth/ISL/

The more advanced text is also available for free: http://statweb.stanford.edu/~tibs/ElemStatLearn/

i think you should read SVM:

http://www.mathworks.com/help/stats/support-vector-machines-svm.html

It is based on your  dataset. If the dataset has high variance,you need to reduce the number of features and add more dataset. After that use non-linear method for classification.

If the dataset with low variance ,use linear model.

In OPENCV SVM have the auto train method,it will detect the classification technique and parameters automatically.

A mehtod is linear (very basically) if your classification threshold is linear (a line, a plane or a hyperplane). Otherwise, it is non linear. Obviously, linear methods involve only linear combinations of data, leading to easier implementations, etc.

In principle, both ANN and SVM are non linear because they use, in general, non linear functions of the data (the activation function in ANN or the kernel in SVM are typically non linear functions). However, in both cases you could use linear functions in the problem is linearly separable.

In most useful cases, a non linear techniques is required, but a linear one is desired... You might use a suboptimal classifier (linear) if the error might be assumed in opposition to the complexity of a non linear implementation.

Hope this helps.

Dear All thanks for Reply.

Dear Rafael, on what basis we can say that a function is nonlinear in nature.

Hello,

Simply, transforming data feature to stable space. Technically, non-linear methods transform data to a new representational space (based on the kernel function) and then apply classification techniques. Most of the time, this transformation describes the data features in a more clear structure in comparison with the original space, which the classification algorithms can create more accurate predictor in the new space. So, if classification techniques reform (transform to new space) data features before applying classifier, most of the time, we call them non-linear methods.

Dear Aman Singh

Look for this page to see nonlinear activation functions used in the ANN

https://www.google.com.eg/search?q=sigmoidal+activation+function&biw=1014&bih=421&tbm=isch&tbo=u&source=univ&sa=X&ved=0ahUKEwim-NH-oM_KAhXLWRoKHXSGATQQsAQIMg

For more detail you can see ANN book like 

Neural Networks - A Comprehensive Foundation - Simon Haykin

Dear Manish,

Thanks for answer. How 3D plane make a difference. Kindly explain in detail.

Linear means the output y is the linear combination of feature x. So, LDA is linear, Support vector Machine using linear kernel is also linear, the NN that use linear combination is also linear.

H-SVM and S-SVM are LDF. My papers evaluate eight LDFs. See my papers and download those from RG.

Hi Aman

We use Linear and non-Linear classifier under following conditions:

1. If accuracy is more important to you than the training time then use Non-linear else use Linear classifier. This is because linear classifier uses linear kernels and are faster than non-linear kernels used in the non-linear classifier.

2. Linear classifier (SVM) is used when number of features are very high, e.g., document classification. This is because Linear SVM gives almost similar accuracy as non linear SVM but Linear SVM is very very fast in such cases.

3. Use non-linear classifier when data is not linearly separable. Under such conditions, linear classifiers give very poor results (accuracy) and non-linear gives better results. This is because non-linear Kernels map (transform) the input data (Input Space) to higher dimensional space( called Feature Space) where a linear hyperplane can be easily found.

For details read the following paper by C J Lin.

https://www.csie.ntu.edu.tw/~cjlin/papers/kernel-check/kcheck.pdf

NOTE: All these points hold good w.r.t. SVM and I am not sure about other classifiers.

Thanks

Vinod

Thanks Vinod.

Hi let me try to explain with small example using regression.  Lets you are fitting a model y = a + bx it is a linear model , where as if you fit a model y = a + bx + cx2 .  Where the non linear model will give a better fit , but you will loose on interpret-ability. 

Thanks Saptarsi. Nice Example.

Nice explanation!

I am learning Data Science/ machine learning. I would appreciate if anyone give intuition as in which algorithm (SVM, Logistic regression, Decision Tree, KNN) should be used basis type of data. 

If we have a multiclass classification problem where there is huge class imbalance then what should be the approach.

Check my nine abstracts of "New Theory of Discriminant Analysis afterR.Fisher" on RG.I compare eight LDFs including SVMs by several datasets including microarray datasets. Eight LDFS can validate several data. But  ANN and kernel SVM cannot validate several datasets. So, two types results are completely different. Later results by ANN and kernel-SVM are less reliable than former.

awesome explanations by all indeed, cheers to all.

Very good explanation with details

Article Linear vs. quadratic discriminant analysis classifier: a tutorial

In machine learning, models such as y = a + bx + cx^2 are still considered as linear models because y has a linear relationship with the model parameters (a, b, c).

for linear equation the line straight in graph, i.e., slope of a line is y=mx+c

for non linear equation the line in curved like position

Here you can read about linear and nonlinear SVM

https://www.reneshbedre.com/blog/support-vector-machine.html