Hi,
I currently work with sensor data (multiple sensors) and I want to be able to classify the time series. At the moment I am using a CNN approach and I am encoding the time series as images. To do this, I am normalising each input sample but separately, meaning I do not consider the max of ALL the sensor data at the moment. The reasons for not doing this are the following:
1. I have several Gigabytes of data that can potentially be teras and it is impractical to look for the maximum value.
2. At the time of training, i cannot be sure I have seen/captured all the possible patterns of time series, so future sensor readings could have a different max value. (concept drift).
Now obviously the problem with normalising by data point is that I lose information of the scale, so at the moment I have images that look very similar, but belong to different classes because if you look at the raw data, the time series "looks similar" but actually the range of the values is completely different. This is confusing my CNN as i am feeding similar images but giving it different labels.
has anyone had this problem before?
I was thinking of additionally coding into the input image the actual max value of the original raw data.
Or could I simply try and provide the raw data without normalisation or image encoding?
thanks for your help
giovanna
Hi Giovanna,
Hope you are doing well.
Regarding your question, I recommend the first option of adding the scale information to the feature vector. You can also avoid feeding raw scale value to the CNN by normalising the scale input value to a reference point (you can set empirically or by data observation). This approach will preserve the time series information.
Cheers
Tariq
- Badges
- Science topic
- Similar topics
- Methodology
- Sensors