I would like to make a model using sequence datasets.
As I've searched that , many LSTM, RNN tutorials use long-sequential data such as weather in 1950-2020.
However, my datasets are short sequential data but concatenate data. I attached a figure.
1) The labelled datasets are collected from 100 students. It has ground truth and the W vector (W1-Wt) can be used to infer the ground truth. The t is less than 50 (short sequence).
2) After fitting a model, I will validated it by showing confusion matrix.
3) The fitted model is used for unlabeled data. This is collected from 10,000 anonymized people. I will use W vector (W1-Wt) of the 10,000 people for inferring.
In this case, which model is good for me? The LSTM, RNN or HMM still have benefit for me?
It depends on train size, time-series length, number of classes (BTW, how many classes do you have?). Using LSTMs requires a lot of data. Since your dataset consists only 100 labeled sequences, and a part of them you should use as a test (at least 10), and sequences are quite short (t
Olga Radyvonenko Thanks for your answer. The number of classes is 2 (stop/move in the case). I totally agree with your idea. How about HMM? I thought that the model dealing with the sequential data should be adopted.
HMMs assume that the states have been generated by a first-order Markov Chain process, and that each observation has been generated based only on the paired state. Thus HMM would be helpful, if you expect that some structure exists in transitions from one segment to another in your time-series. If so, before training you should make a decision on a number of states in you Hidden Markov Model. But if the transitions between time-series segments are random or governed by a non-Markovian process, HMMs will not help.
- Badges
- Science topic