I have not really worked with panels, but in general, the estimated variance of the prediction error is made up of a part for the model (coefficients), and a part for the estimated residuals. (The former also involves the latter.) The former becomes smaller with larger sample sizes, and the central limit theorem applies to estimating the coefficients. The difference between the former and latter parts is a difference between standard errors and a standard deviation. If the latter is a big part of your estimated variance of the prediction error, then your prediction interval will not be in a t or normal shape, and I suppose rather skewed, which may make it harder to interpret.
Not having close to normally distributed residuals then impacts your prediction interval. I think it also may make it not the best of all predictions, but I'm not so familiar with that.
First look at the residual plots by repeated units not the overall plot. My guess is you won't have enough points on each plot to decide normality. You can probably get a sense.for issues like outliers. When you get this information let's talk again if needed. Best wishes David Booth