Dear colleagues,
I am trying to design a recurrent neural network to predict patients' length of stay as output. I have different types of data (numeric, categorical, free text, etc..) and I want to present the data as a time series( to capture the temporal dynamic of the system). It means I want the model to know when each piece of information is collected.
Here is my question:
1- What are the methods to feed the data into the model?
2- What are the advantages and disadvantages of each method?
Dear Ali Kamali Mohammadzadeh
I have a research paper presently under review relating to your question. In my opinion, I would suggest that you have 3 inputs and one output to your Artificial neural network (ANN); the inputs being monthly temperature, monthly relative humidity and the corresponding months. Your output would be the monthly electricity consumption. One way of coding the months would be say Jan as ‘1’, Feb as ‘2’, March as ‘3’......Nov as ‘11’ and Dec as ‘12’. Note that including corresponding months as input attributes make much sense considering that the different months significantly influence electricity consumption. Going further, you will have to normalize the 3 inputs to the range 0 to 1 before feeding them as inputs to the ANN.
Input data coding
Note that you will have to normalize the 3 inputs using the range (or maximum) value for each input correspondingly. Never use a single value to normalize the 3 different inputs, as you create bias in the structure of the data. For example, to normalize the month inputs, you could use the maximum value '12' to divide all months input attributes. You find a suitable value for the temperature and relative humidity too, and then proceed with normalization just as for the months input data.
Output data coding
Method 1: You may normalize your desired output data (monthly electricity consumption) to the range 0 to 1, in which case you'd been using Log-Sigmoid activation function in your output layer. If you choose this method, note that you would have to rescale the outputs back to actual values when you test the trained network. I was absolutely fine with this option in my paper.
Method 2: You may use the monthly electricity consumption as they are (actual values) as desired outputs and then use a linear activation function in your output layer. Note that this option may make learning extremely difficult.
Training and Testing ANN
After you have successfully preparing the data. i.e. normalization of inputs and may be output; you'd want to divide your data into training and testing data; optionally, you could have a validation data. Also, select a suitable ration for training data: testing data. One more important point, in order to reduce the chance of the ANN only memorizing (over-fitting) your training data, you'd probably want to randomize the training data first before training.
Hidden layer
You will have to heuristically determine the suitable number of hidden neurons for properly learning the task. For example, increasing the number of hidden neurons gradually. Also, note that the activation function used in the hidden layer should allow the learning of non-linear functions; hence, you could use the Tangent-Sigmoid or Log-Sigmoid function, though I would strongly recommend the later in your situation.
Other network training heuristics
You still have to determine parameters such as learning rate, momentum rate, maximum number of epochs, required MSE (error).
Karthika .P
This answer was originally provided for another question on ResearchGate. Here is the link to the original answer:
https://www.researchgate.net/post/How_do_I_represent_input_variables_for_artificial_neural_network_design
To me, the very disappointing point is that I think you did not even read the answer that you copied. Because obviously it is trying to address a different question.
In the future, if you come across to a question that you think is already answered on another web-page, I suggest you provide the link to that web-page or cite the reference. I believe this approach is more consistent with the ethical values that we all revere as scholars.
Bests,
Hi,
I think you can standardize the encoding of different input data into one type to make it easier to organize your network, for example you can use Binary coding for every input (numeric, categorical, free text, etc..) may be first input need 4 bits, second input need 3 bit and so on then the input vector compose of the total number of bits for all your different inputs.
Dear Najla Matti
Thanks for your response.
If I understand correctly, you are suggesting that if a specific piece of information (say, a lab result for instance) is collected at 6 PM, I assign a few bits (say, 32 bits) to the result itself as well as a few more bits (say, 16 bits) to the time the information is received. Then, I combine them together (resulting a 48 bits piece of data) and feed it to the network as a single piece of information.
Do you think this approach can capture the temporal dynamics of the system? I am afraid it might lead to learning incorrect patterns by the network. Have you implemented such approach yourself? and if you did, how accurate the results were?
Bests,
- Badges
- Science topic