The primary problem with neural network is that it over-fits the training set. It builds a complex model that estimates the outer layer of the training set almost perfectly, but yields poor prediction of a test set (different from training test). With increasing complexity, predictive power falls. As your model becomes less complex, it will produce more error (validation loss value), but will predict better.It is a trade-off between 'Bias' (More complex models are less biased on training set, but have less predictability on test set) and 'predictability' (Less complex models have more predictability on test set, but are more biased on training set).

The model that does better on the test set has a better generalisation. Test sets represent the outside world not seen at all by your model. Training set and validation sets are the Lab sets.Training set is used for training the model while validation set for tuning the model. Both are used to obtain good generalisation on datasets not seen during this process(which is the test set). Our interest when we train a Neural Network is to minimise empirical risk on the test set. Low error on the training set does not necessarily imply we have got the best model... note that it could be possible that we have an overfitted model with minimal error in the training set. In the other hand, having a low error on the validation set could be that we have so-tuned the model parameters that it overfits the validation set. However, just as Somenath Chatterjee said the best performing model may not be a very complex model. The model that will just work lies somewhere in-between high complexity and low complexity. Make sure you visualise your errors on your different set splits- training, validation and test sets. That would guide you best on what model to choose.