Obviously, the problem is not the dimensionality (11-D is a moderate dimensionality); the amount of data that you chose to handle, and the complexity of the model that you chose to use, are the problems.

How do you know the points that you must use for training? Which experimental planning method did you use, that dictated the choice of these points?

Why do you want to use a neural network with two (why two?) hidden layers and 30 (why 30?) hidden neurons per layer? What model selection method did you use, that showed that this was the optimal model complexity given the available data?

Of course you must forget about Levenberg-Marquardt with such amount of data and such model complexity. Note, however, that the choice of the training algorithm has nothing to do with the accuracy of the model. The key issue is the selection of the appropriate model complexity, and, for neural networks, the initial values of the parameters.

Supercomputers may be one solution. Another, probably much better, solution may be to give some more thought to your problem, especially to revisit critically your assumptions, in order to make the task manageable.

Dear Dr. Dreyfus

to be more specific, I want to solve a PDE with neural network:

Article Artificial neural networks for solving ordinary and partial ...

1- in the first step I solved the problem and the solution of PDE was a 2-D probability distribution p(x1,x2,t) of time.

for such problem, the results was not good enough with one-hidden layer network. When I used two-hidden layer, each with 20 hidden neurons, the results was satisfactory. also with 30 collocation points in each dimension x1, x2, t (30*30*30 samples) the Levenberg-Marquardt algorithm was used.

in the second step I want to find the solution of PDE, that are also the function of some control variables, and initial conditions, in this case,

I should approximate p(x1, x2, t, Xi), i=1,2,...,8.

where x1 ,x2 ,t are the same as before, but 8 other parameters are added which totally give the 11-D function approximation problem.

so I will try to answer your questions in order to further clarify the subject.

" How do you know the points that you must use for training? Which experimental planning method did you use, that dictated the choice of these points? "

based on the first step of simulation, for x1 ,x2, t the tensor product of 30 equal distance points in the domain of x1, x2, t (totally 30*30*30 samples) gives good results. so I know that for good approximation I should choose 30 equal distance points in the domain of x1, x2 ,t. for other parameters (Xi,i=1:8) intuitively for covering all the domain of these parameters, I know I should choose at least 8 equal distance points in the domain of each parameters.

for example if we want to approximate f(x,y)=sin(x)*cos(y), x,y∈ [a,b]*[c,d]

we choose equal distance points in [a,b] and [c,d] and use tensor products of them in order to form the input output data set for training

"Why do you want to use a neural network with two (why two?) hidden layers and 30 (why 30?) hidden neurons per layer? What model selection method did you use, that showed that this was the optimal model complexity given the available data?"

with prior knowledge that I gathered in my first step of simulation, one-hidden layer network was not sufficient so I choose two hidden layer network. Because 20 neurons in each layer gives a good results in the first step of my simulation, for the second step, I thought with growth in the dimension of the input (3 to 11) I should increase the number of hidden unit in each layer (20 to 30).