Is it: (i) simply the elegance as seen by the eye, or (ii) should it be seen from the context of how well it aids scientific research, i.e. its applicability in real life! I am tempted to think the two are intricately inseparable.
First, we must be aware that such beauty is only perceptible by the (applied) mathematician, so we cannot give an absolute definition of it. I think "elegance" is a better term (than "beauty"). In searching for a structural definition, it depends on whether we talk about the elegance of a proof or the elegance of a mathematical model. The former deals only with the mathematical flow of itself and should include criteria such as simplicity, length, walking through several theories and a certain mathematical aesthetics only visible and measurable by math persons. The latter assumes both a proof and a representation of a physical system. For the representational function, we can only take objective criteria related to how well the model captures the relevant features of the physical system and what is the the error range of the results, coming from the idealizations. In conclusion, my answer is: a proof may be elegant in its own right, regardless applicability. Generalizing, mathematics can be beautiful in its own right as creation, even if not applied. A mathematical model can be elegant only if adequate and this is where your (i) and (ii) work together. If we talk about the beauty of mathematics as a whole, take all literary quotable definitions of mathematics. If God is a mathematician - to paraphrase a bestselling title - and reality is representable through mathematics, since reality is beautiful, mathematics should also be so.
A real true mathematical model is by itself is beautiful because it is not easy to model something. After a number of iterations such a beautiful thing is created which can be used for the benefit of mankind. When we model something a part of solution ( essential features) is already known and we investigate further to know more. Sometime it may surprise us because it may not be as per our expectation.
I would personally go for the option that the model can be beautiful even without it being so useful. I have been involved in the creation of such models - or, statements that characterize, for example, conditions for a primal-dual vector to be optimal in some optimization problem. While being in some way beautiful, this particular case has so far turned out to be very difficult to utilize algorithmically. And we must remember that there are MANY cases of beautiful theory from the past that have only recently been utilized - often in surprising and new contexts. So, to go back to the question, certainly the item (i) can come long before (ii) does, and that does not mean that (i) was not beautiful before (ii) emerged.
Artur, indeed symmetry and beauty w.r.t mathematical models are inseparable. It is a marriage that many who appreciate the fun in algebra and geometry reject with difficulty.