I will recommend to visit this statistics blog: https://statisticsbyjim.com/
In brief: the standard error of the regression (S) is valid for both linear and nonlinear models and serves as great way to compare fits between these types of models. A small standard error of the regression indicates that the data points are closer to the fitted values.
The most common techniques seek to minimize the sum of the squares of the residuals; however, there are times when you may be most concerned with the worst case (called the min/max problem) or perhaps the error at some particular point or even bases on some weighting function. This is a broad and interesting field of study and not a simple matter.