I don't think there is 1 text where you will find all these topics. I think at least 2 texts might also be a stretch.
But Nocedal and Wright's Numerical Optimization and Boyd's Convex Optimization book will cover most of the topics you mentioned apart from ADMM and ERM. And Boyd's Convex Optimization lecture II covers ADMM.
I think for ERM, you will have to pick up a Machine/Statistical Learning book.
And I think this is a pretty good deal. Just 2 books and 1 course for everything apart from ERM.
If someone else knows better than what I wrote, I would also be interested in knowing this information.