I suggest you to use Deep learning with Artificial Intelligence with famous architecture for example NN, CNN, RNN....

we get you better accuracy and high predictions when you testing.

I hope that be Clair for you and helpfully. @TzeHuey TAM

In general, the SVM is more popular and easy to implement. Random forest is also very old passion algorithm popular for numerical data. There is no such algorithm that may be best in general for every data. The suitabilities of most algorithms is just variational w.r.t. accuracy or error rate over different data.

I suggest you to use Deep learning with Artificial Intelligence with famous architecture for example NN, CNN, RNN....

we get you better accuracy and high predictions when you testing.

I hope that be Clair for you and helpfully. @TzeHuey TAM

Basically, SVM is great for binary classification, but for multi class classification RF is better

SVM has several variations and RF is a kind of decision tree learning method. In general SVM usually lead to better accuracy due to optimal margin gap between separating hyper planes, which make it could do predictions better with test data. Meanwhile , RF is mainly used for multiclass problem. Besides, the results are very much a dataset-dependent problem.

First of all, your question is contradictory:

1. "Support Vector Machine (linear kernel) and Random Forest has _comparable_ accuracy, any reason?"

2. "What is the main to explain RF has _lower_ accuracy or SVM linear kernel has good accuracy?"

So as others pointed out, ML algos are not magic weapons. If you have a correctly formatted input data and the features in the feature vector, many algo will go you comparable, good performance. In general, I would add that:

- SVM is very sensitive to the data vectors. You must normalize to [0; 1] with mean-stdev. However, you can lose some information during that transformation which can lower the prediction performance. On the other side, linear SVMs are quick to train, they have a very low memory footprint and they are quick in inference.

- RF is not sensitive to the data vectors, you don't have to normalize them. The disadvantage of RF that it may overfit the data in training easier than SVM and you can run out of RAM very easily in case of a bit bigger datasets if you don't specify the growing hyperparameters carefully (max tree count and max tree depth). On the other hand, giving too low values to these latter parameters can lead to low accuracy since the forest can run out of capacity during training.

In the end, I also advise trying some simple CNN or MLP on your data. Whatever it is numerical or image. Even small DL models can lead to surprisingly good results on some problems.

Dear TzeHuey,

The random forest (RF) algorithm has been successfully used in the past, providing accurate land cover maps SVM is often claimed to be the best at dealing with complex classification problems such as tree species differentiation, followed by RF.

Thanks

SVM can give you better solution in comparison with RF, because SVM produces the best possible solution for your current data automatically. The SVM method can guarantee this. RF needs some subjective choices from your side: the number of trees, the complexity of trees, etc. That's why sometimes SVM can do better, because of the mathematical power of the method. In some cases, when your feature space is nearly linearly separable, SVM does extremely well, but RF can degrade in this situation. Besides, SVM is a so-called "sparse kernel method",

which means that the resulting inference is low memory consuming and very fast. RF inference can be quite 'heavy'. But RF is a 'committee method' and that means that potentially you can achive better accuracy by making your

committee more complicated. Also, RF can be better in the case when your feature space is extremely non-linear.

Muhammad Ali Andrey Dashkov Ankur Srivastava Csaba Kertész Farzana Kabir Ahmad Musaab Alaziz Barodi Anass thank you for your suggestions.

The two model (SVM and RF) may have the same accuarcy but this doesn't mean that they perform at the same level. I suggest to check the paradoxe of accuracy and the trade-off between accuracy and variance.... to really know which of them has the best performence. In addition, as pointed above, Machine Learning is not a panacea, it is a dataset-dependent problem.

You seem to be surprised to find better results with linear SVMs than with Random Forests. Your surprise is probably due to the fact that Random Forests (which have been around for a number of years) have recently become the trendy thing to do. SVMs too were the trendy thing to do in the 1990s and early 2000s, and hundreds of publications declared all other classifiers (including neural networks) dead. This phenomenon happens all the time in machine learning, despite the fact that it was proved in 1996 by Wolpert et al. that no machine learning method can be uniformly superior to all others on all problems (the so-called “no-free-lunch theorem”).

First of all, you should ascertain that your observation is not an artifact due to the partition between the training/validation set (containing the data that you use for training the models and for finding the “best” hyperparameters, e.g. by cross-validation) and the test set (containing data that you never used, neither for training nor for hyperparameter selection). In order to ascertain that your observation is not an artifact, you should do the following experiment: apply exactly the same procedure for different (say, 50) partitions of your data between training/validation set and test set; then, compute the means and standard deviations, over the 50 partitions, of the accuracies of the 50 SVMs and of the accuracies of the 50 random forests thus obtained; then, perform a statistical test of the difference between the means. If the hypothesis of equal means is not accepted with the chosen confidence level, you will conclude that the results of that experiment supports the hypothesis that the two methods perform identically on your data, which would not be a big surprise. If the hypothesis is rejected, you will conclude that the results of your experiment do not support the assumption that the two methods perform identically on you data, which would not be a big surprise either. In the latter case, it might be worthwhile investigating the reason for the difference.

The first explanation that springs to the mind is the following (but there may be several others). Linear SVMs have either no hyperparameter at all (hard-margin linear SVMs), or a single hyperparameter (the regularization constant of soft-margin SVMs). By contrast, Random Forests have roughly half a dozen hyperparameters. If the examples of the training set are linearly separable (this can be ascertained in finite time by running the Ho-Kashyap algorithm on your data), a hard-margin linear SVM is guaranteed to find the optimal solution given your training data. If the examples are not linearly separable, you can try to use a soft-margin linear SVM, at two costs: (i) optimizing the hyperparameter, and (ii) losing the guarantee of optimality: the solution will be optimal given the available training data and given the value of the hyperparameter. Alternatively, you can try out another linear classifier such as logistic regression if your data obeys the assumptions underlying that method. If the soft-margin SVM does not provide satisfactory result despite your efforts at optimizing the regularization constant, or if the data do not comply with the assumptions of logistic regression, you may resort to a multilayer Perceptron, which has a single, discrete, hyperparameter (the number of neurons in the hidden layer), and possibly a continuous one (the regularization constant). This covers a fairly large area of applications. If none of the above methods provide satisfactory results, you might have to resort to more complex models; in so doing, you should always consider carefully the number of hyperparameters involved in the methods: the smaller their number, the lower the probability of overfitting the model to the data; in addition, finding the optimal combination of continuous hyperparameters is very difficult (maybe you just used the “default values” of your toolbox and did not perform any optimization).

In short, do not jump to complex, trendy methods unless you have tried out the simple methods and failed to obtain good results.