Hi Safae,

I'm a little puzzled by your question "how can we use cross validation in weights determination", in fact I find it really strange. What exactly do you mean? As far as I know weight determination and cross validation are quite orthogonal matters. Can you develop your idea/question, please ?

Also, it it not clear to me what you mean when you say that "K cross validation generates k models". I disagree with that. With only *one* cross-validation data set you may generate an infinity of models, if you agree that initial weights are random and that 2 networks that are identical as as to their architecture and parameters but have different initial random weights are different (this is something most people -- not to say everybody -- agree on, because there is literature that shows the huge influence that different initial random weights have on training dynamics and its result).

Let me know what you have in the back of your mind, I'll be glad to help.

Cheers,

SCM

Hi Serban,

First, thanks for your attention. I agree with you on all what you said.

Concerning my problem, I want to use neural networks for prediction but I have some difficulties to implement neural network models.

If I suppose that a trained neural network model consists of the neural network structure (or architecture, I mean number of inputs and outputs, number of hidden nodes, learning rate ...) and the weights, then I have to determine the best structure and after determine the weights through training.

I've done some researches about the process of neural networks elaboration. I found in some websites that K cross validation can be used in the training stage in order to deal with overfitting and to find the best values for hidden nodes and for the structure parameters. After that, we can finally train the network using all the training data. (http://jamesmccaffrey.wordpress.com/2013/10/25/k-fold-cross-validation-for-neural-networks/)

In general what is the best process of neural networks elaboration ?

In the case where the neural network structure is already defined and I want just train this neural network in order to determine weights using my train data, how could I proceed? Can I use the k cross validation process in training stage for this case ?

Cheers,

Hi Safae,

First you'll have to chose, on theoretical grounds the "right" type of artificial neural network (ANN) for your problem. There are quite many ANNs and in order to chose right, you will have to do some reading on this topic.

The second step is deciding what training algorithm you'll use with your ANN (dependind on the ANN type, you may have to chose among many, or, to the contrary, in chosing the ANN type you'll also have chosen the training algorithm). You will most certainly understand this better once you will be done with the reading I allude to in the previous paragraph.

And then, and only then, you need to think about cross-validation. This is a refinement on top of all the previous things I mentioned, not a method (i.e., an algorithm) to acheive learning. Cross-validation is a method to ensure learing is sound if and when learning occurs.

Here is a post of mine about cross validation (with K=3):

https://www.researchgate.net/post/why_the_prediction_or_the_output_of_neural_network_does_not_change_during_the_test_phase?

If you are a newcomer to ANNs and you read French, you should read parts number 1.3, 2.3, and 2.4 of my PhD document from here :

http://www.risc.cnrs.fr/mem_theses_pdf/2004_Musca.pdf

HTH. Let me know.

Cheers,

SCM