Simulating protocol security is very much protocol codes, attack vectors, server and database farms network architectures specific. It used to be a custom development.
However, apart from security and privacy issues, the key characteristic of the completed remote authentication solution is its efficiency - how many authentication events the system can handle at worst peak times without malfunctioning during standard mass services. It is also typically a custom stuff, but it is easier in implementation.
You can use the Scyther tool https://people.cispa.io/cas.cremers/scyther/ or the The Tamarin prover https://tamarin prover.github.io/manual/book/001_introduction.html in order to verify the security claims of the proposed protocol.