There are various types of neural networks, with artificial neural networks (ANNs) being among the most prevalent, constituting classical or dense neural networks. Another widely employed category is convolutional neural networks (CNNs), particularly effective in image analysis. Recurrent neural networks are extensively utilized in natural language processing (NLP), and it is within this domain that additional network types have emerged as advancements over recurrent networks. Notably, the Long Short-Term Memory (LSTM) networks and Transformers have gained prominence in NLP applications