i want to work on text classification using deep learning. please suggest me best baseline.
Hi Prashanth,
Can you give more details about your text classification ?? what do you want to achieve ?? which dataset ?? how many features??. If you're trying to get started then Naive Bayes (MultinomialNB), SGDClassifier are the best. Deep learning (Lstm) would any day be my choice. You have to do a lot of cleansing, hyperparameter tuning and build the pipeline.
Regards,
Athiq
Deep learning is about ANN being able to have to more hidden layers. More hidden layers means processing capacity requirements are exponentially high. Since the introduction of GPU computing, deep learning has become reality, although ANN are there since 60+ years.
In your case of text classification, you dont need to use deep learning unless warranted. If you share the scenario of text classification, one can suggest the method.
One example that demand deep learning for text classification , to my knowledge could be the News service by broadcast channels.
Do come back with more details
Best wishes.
Kindly have a look at the following best research articles on Deep Learning for Text classification:
https://towardsdatascience.com/best-research-articles-on-deep-learning-for-text-classification-2015-2016-aaa7950af775
Hinton:
https://adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html
Kindly have a look at the following details regarding base line learning and implementation of deep learning:
https://becominghuman.ai/expensive-lessons-i-learnt-during-the-implementation-of-my-first-deep-learning-network-aa9a5da2331a
http://deeplearning.net/tutorial/deeplearning.pdf
I hope you have a target for classification, set up a sequence model (Lstm would be ideal) and also do WordNetLemmatizer, CountVectorizer together with tfidf. Once a sequence model is set try hyperparameter tuning!! example as below !! it is very intensive !! try with fewer data and one parameter after the another !!
batch_size = [10, 20, 40, 60, 80, 100]
epochs = [10, 50, 100]
optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
learn_rate = [0.001, 0.01, 0.1, 0.2, 0.3]
momentum = [0.0, 0.2, 0.4, 0.6, 0.8, 0.9]
init_mode = ['uniform', 'lecun_uniform', 'normal', 'zero', 'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform']
activation = ['softmax', 'softplus', 'softsign', 'relu', 'tanh', 'sigmoid', 'hard_sigmoid', 'linear']
weight_constraint = [1, 2, 3, 4, 5]
dropout_rate = [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
neurons = [1, 5, 10, 15, 20, 25, 30]
param_grid = dict(neurons=neurons, dropout_rate=dropout_rate, weight_constraint=weight_constraint, activation=activation, batch_size=batch_size, epochs=epochs, optimizer=optimizer, learn_rate=learn_rate, momentum=momentum, init_mode=init_mode)
grid_search = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, scoring='accuracy')
From my experience with text mining the pre-processing steps are at least as important as choosing an appropriate DL scheme. Depending on what you are mining for, a simple bag of words approach might be sufficient to get a high classification accuracy. E. g. a bag of words can be used as input vector for a ConfNet. More sophisticated approaches such as TF-IDF or tokenization might yield better results for some classification tasks. If you have a large body of documents you might consider clustering as preliminary step to improve performance. Especially with DLs not only the predictive power is important to evaluate an appropriate DL method but also calculation time and ease of interpretation of the output.
- Badges
- Science topic