Hi everyone!
I'm working on a problem with Video Classification.
I have an algorithm to extract feature vectors of sequencies of 40 frames extracted from 4 different video classes.
I'm using AlexNet to extract these features from these frames and as output from 'fc7' layer, i have a 40x4096 matrix, where each row refers to features of one frame (one frame per row), so i'm passing one frame at time through the AlexNet.
So i need to classify videos between these 4 different classes. I will preprocess a new video to limit its number of frames and then extract features from this video to classify it.
I need to know how to store these extracted features in order to train a classifier and classify the new 40-frames video.
So i have a 40x4096 matrix of features extracted with the "activations()" function of MATLAB from one video, but this matrix don't have any information about my class label.
I have to convert each matrix in a vector of information and use it as a vector of characteristics to train and to be classified by a SVM, for example? (or any other classifier)
A Support Vector Machine can be used to classify videos like in my problem or should i have to choose another classifier?
I'm in the right way, extracting 40 frames per video, extracting features from each 40 frame and obtaining a 40x4096 matrix?
I need insights!
Thanks for the support!
Okay, you have 4 video classes (ci, i=1,..4). Each video has 40 (n =1,..40 ) frames and from each frame you extract 4096 features ([1x4096]). If you think that each frame has enough information to predict the video class (ci) then you can use the 40 frames from each video as your training/test samples, which will create an input matrix INPUT [160x4096]. 160 samples or examples and each sample has 4096 features. Additionally, you will have to create an output vector [160x1] that contains the label of each class ci = i where i=1,...4.
You can start with ANN then experiment with SVM.
Ok, Turki Haj, i'm passing through the AlexNet 40 frames inside a 4D array and receiving as output a 40x4096 matrix.
So, in theory, each 40x4096 matrix represent a 40-frame video.
Should i convert these matrices (40-frame video features) to vectors of characteristics (one vector per matrix)?
Can you please explain what is your goal? Do you want to predict the video class using one frame or 40 frames? How many videos you have?
Ok, i created a repository on GitHub and i will explain about my files.
GitHub -> https://github.com/gonsalesarthur/cnn-lstm-matlab
mainFile.m
In this file i call all the other files from my project folder.
Functions:
- frameWriter('input_dir', 'output_dir', 40)
This function extracts 40 frames per video from the input directory to the output directory, so for each video listed in the input directory, the function will generate 40 frames in the output directory.
- stackArrayGenerator(path)
This function stacks inside an array of 4D a total of 40 frames from the directory (path) and return a variable called "array4d" that contains the array of images.
So i want to predict the video classes using all the 40 frames.
Extracting features from 40 frames:
I'm using the AlexNet Convolutional Neural Network to extract features from each frame.
Code here:
net = alexnet;
layer = 'fc7';
features = activations(net, array4d, layer, 'OutputAs', 'columns');
The variable "features" stores my 40x4096 matrix of features, where each row represents features from one frame, so 40 frames = 40 rows.
I need to use this features to train a classifier to, obviously, classify the 40-frames new videos.
For now i'm using the UCF-101 Action Recognition data set, but after it's works fine, i will adapt this model to video data of the laboratory where i'm working. :)
My first idea is to use these features to be classified by an LSTM model, because i think it's the best model to classify this type of data, but if you recommend me another way to do this, i will consider. :)
That's it! :)
- Badges
- Science topic