Z notation is quite old. For validation you still can use ZEves tool, can be integrated with CZT-IDE. CZT-IDE also provides type-checking for Z specifications.
There many others different formal languages and tools can be used; can discuss if you provide more info on your problem of "DSL formalisation". Please note, EBNF is the typical approach for a language development.
I'm not sure if I got your question, but you could have a look at some works on software design using Alloy or Spin. Alloy, for example, provides a formal specification language and tool support to run some analyses on the specs.