What is the porpuse of this modification from the common multilayer networks? 

Imagine that for classic Feedforward networks, synaptic weights of lateral connections among hidden units are equal to zero. Then, isn't the latter more general? Specifically, I would like to implement this kind of network in Matlab, but how can I express the output of hidden units with lateral connections?

Classic neural nets do not have lateral connections. This creates a situation where members of the layer are interdependent. I don't believe this can be easily solved with standard back propagation. Have you thought about putting your data into a graph representation. As Juan implied though, we would need to know what you want to do to help much. 

Maybe, you have to propose an algorithm or new programing to the training method. Neither NeuralLab nor Matlab consider internal conections among neurons of the same layer. The development of the new program can take you so much time, is not easily solved.

I haven't given it a great deal of thought, but I think simply adding additional layers with connectivity rules that match the architecture that you want should approximate what you are describing.

The lateral connections don't present any new issue of principle. The computational method remains the same, as in their absence; the equations, that represent the associations, simply, acquire additional terms. 

 Thank you for your answers!

What I'm trying to do is to write the Matlab code of a multilayer perceptron (2-2-1) with lateral connections in its hidden layer. I recently used Differential Evolution algorithm to train the same network (but with no lateral connections) in order to solve the eXclusive-OR problem (you may find the Matlab code). If you check this out, you can see that the output of the first hidden node is given by:

yco1 = logsig(pop(1,1)+ pop(1,2)*input(k,1)+pop(1,3)*input(k,2));

where pop(,) denotes the element of the i-th row and the j-th column of the matrix pop (the population of synaptic weights. Just imagine that "pop" means "w" in classic notation)

Now, how can I modify this code to include lateral connections between hidden units?

one Hint,

We can also calculate Perceptron like Hopfield Network, with each note calculated as distinct perceptron. Naughty but works. :)

But, the lateral line you speak shall calculated as n * (n-1) lane, 2 in your case. :)

Erick, as far I understood your request, this is no more feedforward architecture. So you may consider recurrent neural network architecture.

Thank you all for your answers,

Let´s consider the image attached below, but excluding the lateral connections (red arrows), the output yielded by one neuron at the hidden layer is given by:

yh1 = logsig(w11*input(k,1)+ w12*input(k,2))

Now, if we take into account the red arrows, how can the expression above be rewritten?