I guess you are looking for tools for automated protocol analysis tools? Because the range of (non-automated) formal security analysis tools and models (simulation- based or game-based) is huge.
Regarding automated security analysis: the following tools/theorem provers come into my mind:
ProVerif
Tamarin Prover
F*
Isablelle
DY* (for the Web)
Just to give you a starting point for your literature research.
Mustafa: I have brought the most recent paper that survey the new practical Formal Methods for Security, for you, its 2021 paper, I hope this will help you a lot