If I have three data sets and want to see how well rules can be elicited from each data set, would it be best practice to take an off-the-shelf pre-trained model and train it on the three data sets, or to use something like Keras Tuner and develop a model from scratch for each of the data sets?
My current thinking is there could be some variance in a pre-trained model's ability to adapt to each data set, leading to bias, and so the way to control for that variance is to train a new model using Keras Tuner for each of the data sets (with all the tuner parameters being controlled).
Any one have any thoughts?
My opinion is to develop ML model from scratch and tune it so it can give you as best as possible output results, plus it's nice additional information for research papers.
Just as Konstantin said, you can train your own model on your datasets, fine-tuning the parameters to get the best results with the given data. This could be the baseline of your work. After it, maybe try to use the pre-trained one to either initiallize the model you got from the last step or to use it only.
Transfer learning makes sense, in my opinion only when the data has some potential similarities (not hundred percent, but homogeneous at least). So in this case, we should know what the best results as so far are with transfer learning.
Then we can do any strategy to shrink the variance if the off-the-shelf one helps.
- Badges
- Science topic