In my masters' work, I noticed the same. Although I didn't mention this in my dissertation, I am curious to know the reason.

As per my understanding, when you enforce two regularisations (L1 and L2) in Elastic Net, you are overconstraining the problem often resulting in an ill-posed problem even though the coefficient matrix is not ill-conditioned.

For example, let us say the L1 regularised loss is bigger than the L2 regularised loss in each epoch and due to spectral bias you would end up fitting data on bigger loss (dominant frequency).