As far as I understood, you are essentially asking how to estimate the global minimum of the loss (RMSE in this case) based on the model's characteristics.

Unfortunately, I don't think there is a way around training the model or looking at the results similar models achieved. Even networks of the same size can perform vastly different depending on the specifics of the architecture.

Cheers,

Raoul

If the model is an Artificial Neural Network then you can consider it as a black box you have no control over the outcomes except what you provide in the beginning as input, training, validation data, parameters, and architecture settings (number of hidden neurons, training algorithm, % of training, validation and testing data).

You can run a model for a specific number of iterations, and you can end it when it is equal to or lower than Mean Squared Error which is the average squared difference between outputs and targets. Lower values are better. Zero means no error.

or

you can compute Regression R Values to measure the correlation between outputs and targets. An R-value of 1 means a close relationship, 0 a random relationship.

Zero, of course.

However the real issue is that this depends on whether the network can represent the model at all. What is, unfortunately, not appreciated is that it is not true that any network can represent any model.

However, contrary to popular belief, it isn't true that nothing can be said or is known; on the contrary, quite a bit is known; cf. for instance, Article Learning in feedforward layered networks: The tiling algorithm

and Preprint Modelling the influence of data structure on learning in neu...

Furthermore, focusing on the RMSE assumes that the distribution is Gaussian, which, for the models that are of relevance for applications, is, in general not true, since the assumptions of the central limit theorem that is, unfortunately, too frequently appealed to, simply don't apply.