Hello. I don’t know where else to ask this. Basically, I successfully extract features from a sequence of images and then concatenate this feature vector with a vector of bounding box coordinates and I put this long vector through a two-layer fully-connected layer to obtain bounding box coordinates of the agent in the future.
I wanted to see whether I can overfit a small subset of my data. So, I use this method on a sequence of just 150 images. I am unable to overfit it, in fact, the loss function keeps increasing. I have done a lot to try to deal with this. I normalize the input to my fc-layer by using torch.nn.functional.normalize(). My fc-layer is a two-layer network that takes a 1028 vector as input and has one hidden layer with size 516 and the output is a vector of 4 values that are added to the input to predict the future location of the bounding boxes of the agent in the images.
Please tell me what kind of problems could I be facing here? When something like this happens, does it mainly mean that there is no sensible relationship between the input and output or could some hyperparameter tuning fix this? (I also increased and decreased the learning rate using Adam without any weight decay)
Things to consider:
- Maybe your gradients are exploding, try gradient clipping
- Maybe the order of your observations are somehow structured, try shuffling your data set
- Investigate how much noise your data has, plot the distribution of the values in your subset vs whole data set
- Badges
- Science topic
- Similar topics
- Vectorization