Without knowing the optimization function you want to solve, it is impossible to give an advise. However, some analysis on optimization function might help you to prefer either of them in advance (the shape of function, variable types, number of objectives, etc). But it is very hard to guess the performance without running trials. By the way, based on my experience, PSO would be the first choice in real-valued search spaces.
I agree that PSO is more versatile (the objective function need not to be differentiable, even once, while double differentiability is a prerequsite of LM method). The "inefficiency" of Levenberg–Marquardt routine may usually be related with problems of finding a good initial approximation of the solution (a single "starting point"). PSO operates on many points at once. On the other hand, PSO is a Monte Carlo -type procedure, a kind of "educated guess", not a purely systematic process. It is usually fast, when you need moderate accuracy, say 1%-5% in objective function value, not necessarily in location of optimum. PSO is "condemned" for success, but only after infinite time. LM method finishes in smaller number of steps and delivers much better accuracy, together with the quite reliable estimates of final uncertainties.