Could you please let me know what kind of neural networks other than recurrent neural networks capable of implementing DFA? How can a neuron be assigned to a state in DFA?
non recurrent neural networks such as TDNN (not recurrent but with a finite delay line as input) have a limited expressivity : such networks can only simulate DFA whose state depends on a limited history (the length of delay line), otherwise known as "definite memory machines"
on the other hand, even simple RNN such as Elman networks can simulate any DFA (with hard limiters activation function ... hard to train ! see Kremer, IEEE Trans. Neural Networks, 6, 4,1995 ; i remember the same is true for NARX but i can't find the reference)
I recommended to read the following two papers about that:
Omlin, C.W., Thornber, K.K. and Giles, L.C. Fuzzy Finite-State Automata Can be Deterministically Encoded into Recurrent Neural Networks. IEEE Trans. On Fuzzy Systems, volume (6), issue (1), pp. 86–79, 1998.
Chandra, R. and Omlin, C. W. A Hybrid Recurrent Neural Networks Architecture Inspired by Hidden Markov Models: Training and Extraction of Deterministic Finite Automaton. In: Artificial Intelligence and Pattern Recognition, 2007, p. 278 – 285.