It is mostly the depth of the neural network. deep NN are considered those with multiple hidden layers although there is no specific number for it. a regular NN could be one with a single layer. the difference between them in respect to their performance is that deep NN show better performance (accuracy...) in multiple tasks. in order for a shallow NN to reach performance similar to that of a deep NN it would require very large layers.