Dear Yali Chu

Try increasing the number of training iterations.

looks like this is similar to freezing certain layers at a time and training

I will answer Q2. I would recommend performing hypothesis tests that can spot any differences better than the human eye to examine if the differences in the performance of the models are statistically significant. One can apply the post-hoc 5×2-fold cv paired t-test or the k-fold cross-validated paired t-test