In Gaussian when I do the 《opt freq M052X/6-311G(d,p)》analysis and some other times with another method and basis sets I face to notice (no special actions if energy rises).
I don't see any problems with waiting a bit longer to continue with the process. This is especially true if the exchange functional (M05-2X) and basis set are higher (triple zeta) in terms of computation. If the molecule size is larger, it usually takes more time, but having enough processors should help speed up the process.