Currently working on sentiment classification of music reviews collected from a review website in real time. Is it feasible to conclude that the proposed classification method performs better (in terms of accuracy) with respect to existing methods (literature) in music sentiment mining, if the existing methods differ in data source used (blogs, forums),data set size and experimental analysis done under different conditions.
There is a huge amount of Scientific publications that approach this exact subject. I used SentiWord for a similar approach, and OpenNLP for stemming, but it depends on what you need.
I think it is better to compare apple to apple. Use the same dataset to compare algorithms. Use the same algorithm to compare the quality of the datasets.
You want the data to be as congruent as possible. Find one or two sources and apply the different methods to them.
In order to say that your method is superior or competitive to other methods, the common approach is: (i) selecting a group sentiment analysis datasets; (ii) choosing one or more performance measures (e.g.: accuracy); (iii) choosing a re-sampling method (e.g.: 10-fold cross validation); (iv) performing a statistical test
See the 6th slide of the following presentation:
http://www.icmla-conference.org/icmla11/PE_Tutorial.pdf
Agree with Mr. Ahmed Hassan Yousef.
You should use the same dataset to compare the performance of the algorithms (e.g accuracy, recall, precision)
Elaborating and emphasizing the earlier comments: Data set used, data set size and design of experimental analysis significantly impact results of classification algorithms. If the algorithm is varied, then it is not possible to explain the cause of difference. Keeping all factors (training data and methodology) constant, variation in algorithms is the only source of difference in results. This will help you to draw better conclusions.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification