Higher order can lead to increased variance of a model. i.e. it will fit the training data better but could Be wildly off when used on new data not in the original training data
It's not always true in general. If we restrict ourselves to numerical approximation of sufficiently smooth solutions then yes a higher order method should be closer to the exact solution than a lower order method for the same mesh resolution in the limit. As pointed out by Harold Berjamin discontinuities in the solution and its derivatives causes sub-optimal accuracy in the method. This is part of the reason why there is such an abundance of numerical methods. Each one has its strengths and weaknesses at resolving various dynamics (e.g., shocks, cusps) associated with the solutions of a given model. If sufficient care is not taken, high order methods can sometimes lead to non-physical oscillations and other types of instabilities in areas with sharp changes in the solution. Chapter 6 of Francois Bouchut's book Nonlinear Stability of Finite Volume Methods for Hyperbolic Conservation Laws and Well-Balanced Schemes for Sources provides some interesting examples of this. Another issue to consider is roundoff error - some methods are more sensitive to roundoff error. Although not a limitation strictly speaking, high order methods are generally more computationally involved and it can be easier to make an error in the practical implementation.
In many cases, higher order discretization is thought of in terms of a Taylor series expansion manifested on a mesh (often 1D in the practical sense). So "accuracy" viewed in terms of the higher order terms in the Taylor expansion reduce in magnitude as powers of the mesh increment. Hence, a claim can be made that accuracy improves. Unfortunately, this view of accuracy is referred to as a "formal" order of accuracy. Particularly, if we move into more than one dimension, formal order does not extend to the real order of accuracy. It has been shown for 3D CFD problems that, for example, a formal second order discretization scheme offers accuracy of about 1.4-1.5. One can approach the formal order only by significantly modifying the scheme to include more data representative of the higher dimensional geometry. Moreover, an even better course is to move toward a discretization that is better representative of the physics. An example is moving from a a Taylor/Finite Difference system to a Galerkin system. Of course, in doing so, one must always pay close attention to the details of the discretization. An old cliche may be appropriate here. "The Devil is in the details".
Actually, the higher-order methods sometimes can even be less accurate, especially for larger step sizes. You can see examples in the attached paper and especially the figure, where the blue and pink lines represent higher-order methods. The order means how fast the error decreasing for decreasing step size, but it says nothing about the error at large step sizes, i.e. from which value and when the error starts to decrease. The round-off errors can annihilate the expected large accuracy at small step sizes, especially for stiff systems.
Article Construction and investigation of new numerical algorithms f...
Dear Tekle and all,
Hello,
Higher order implies faster convergence to the exact solution, or in other words, the capability to decrease the errors when the algorithmic parameter is sufficiently small. To say better order of accuracy talks about the speed of increasing the accuracy close to the exact solution and higher order implies that close to the exact solution the accuracy can be increased faster, which does not represent the amount of accuracy. Therefore, my answer to your question is negative. Methods with higher orders do not necessarily produce more accurate results. This is also very well displayed in the figure dear Endre Kovacshas has placed in his response.
With many thanks for your kind attention have a nice and very healthy day and future.
Not necessarily - see the excellent comments by Harold Berjamin & Robert Miller.
The answer is "not always", with the most common example of high-order doing worse being the case of sharp/under-resolved gradients (e.g., discontinuities) - as already mentioned in the other responses.
I'll offer another example, which is non-linear settings in the absence of sufficient numerical or physical diffusion. In such instances, aliasing errors can become problematic, challenging both the accuracy and stability of the method. Aliasing activity is known to increase with the spectral fidelity of the discretization, which implies that higher order methods are more susceptible. In the case of polynomial non-linearities, however, an aliasing limit can be identified relative to a Fourier basis. Thus the benefit of high-order schemes can be re-established with suitable de-aliasing. We have considered such questions relating to the overall "resolvability"/fidelity of FDMs for smooth flows in the following paper:
Article Balancing Aspects of Numerical Dissipation, Dispersion, and ...
High-order methods also have the tendency to increase the stiffness of the semi-discrete system. So they would require smaller time steps in order to remain within the stability region of explicit time integration methods.
The problem is with terminology. High or low order
always assumes class of the problems - this is what all
previous answers are trying to communicate.
And this is not just class of PDEs but also smoothness of the mesh.
I prefer to use "formally high-order" methods.
Dear Mikhail Shashkov
Very sorry, I apologize for writing this however for a clear scientific discussion, I feel essential to say that what you have written is not what I wanted to transfer to others and specifically dear Tekle Gemechu
It seems to me very clear that the Order of an analysis method (or scheme as stated by Tekle Gemechu ) mostly known as convergence rate (always equal or smaller than the order of accuracy) is different from the order of the problem. For instance you can solve the equation of motion of an multi-degree-of-freedom system subjected to earthquake which is a second order problem by an integration method which is of order four (or any other order different from two). I hope these lines were in the direction of improvement towards better answer of the question and more knowledge for all of us. Have a very nice and healthy day and future.
Hello to all again, In spite of my past answer (two above) I can not my older answer now. In any way, it seems to me that that the higher the order of the scheme the higher is the potential of the computed solution to approach to the exact solution when the computed solution is close to the exact solution. In other words the order of scheme is the maximum rate by which the computed solution can get closer to the exact solution, which maybe different from how close is the computed solution to the exact solution. It also seems simple to conclude that when the order of the scheme is higher it is generally easier to arrive at solutions with higher accuracy. My clear brief response to the question is "No; not necessarily". best wishes for all; have a nice and very pleasant and healthy day.
High-order or low-order scheme is based on a Tailor expansion of the local linear equation, however not all the equations are linear, such as Euler equations, so we need first to consider how to approximate the problem to a locally linearization process. This is done by Riemann solver or flux splitting. However, even we can linearize equations locally, but not every where can we improve the accurate order as high as we want. When discontinuity exists, as proved by Godunov, the linear scheme must no more than 1st-order if stability computing shock waves. Break works are done by TVD scheme, ENO and then WENO schemes, if we construct nonlinear schemes we can bypass the restrict of linear schemes and gain high-order at smooth regions, but in fact, near the shock waves, all schemes must automatically reduce the order if they want get a stable result.
More to say, the linearization process of the equations are also with orders, most of them are first order based on a constant assumption of the Riemann problem. If you want to improve it order, you can use generalized RIemann solver or use ADER method or UGKS.... But they are really more expensive and may cause other problems. First-order linearilization is still the most common used in the study of PDEs.
So just to my knowledge, higher-order may not lead to higher-accuracy.
High-order spatial and temporal schemes can deliver higher convergence rate of truncation errors for smooth problems, but we should pay attention to the global absolute errors as well. In some cases, there exist low-order schemes that can be more accurate than high-order ones when we look at the absolute errors.
Higher-order schemes are generally accurate only when the solution is smooth. Higher-order schemes generally may not show uniform convergence on non-smooth solutions. Even if they show uniform convergence they show a rate of convergence of about one or lower than that in non-smooth solutions.
I have worked on high-order schemes for problems involving shocks using WENO schemes. Though they are theoretically at least third-order accurate on problems having shocks they cannot give third-order convergence. Though higher-order schemes are not showing a higher rate of convergence on problems having shocks they are more accurate than lower schemes for a given error because of high-resolution property.
You can check the following paper where we have shown that higher-order schemes show lower-order convergence on problems having shock.
Article Hyperbolic Runge-Kutta Method using Genetic Algorithm
In the attached figure, I have compared the computational time required to achieve the given error for different schemes like first-order, second-order and WENO-JS (5th-order) schemes on various shock tube problems. Here, higher-order schemes outperformed lower-order schemes, not because of higher-order, it is because of high-resolution properties!
You can also check a small document I prepared about high-order schemes in below attachment on linear problems.
Final answer:
1) Solution is smooth or not? If smooth go for higher-order
2) If the solution is not smooth. Can you put more grids on the solution to make the local (grid points) * (gradients) relatively small? Then go for higher-order
3) If you cannot afford enough grid points go for higher-order high-resolution schemes like limiter, WENO schemes etc. (They are economical and accurate on the uniform grid but may not always be accurate on the unstructured grid)
Generally higher-order means higher accuracy indeed, unless the grid is far too course. This holds even if "high-order" refers to the Taylor series expansion method which holds only for very small grid steps and derivatives not too large. But: In wavenumber space, based on wave-like solutions that are fundamental solutions of the Euler- or Navier-Stokes equations, higher-order methods show typically also superior performance, even if not optimal; optimized schemes freeze the order to four, five or six and optimize the scheme with the remaing parameters in wave space then as for dispersion and dissipation error up to a possibly large wavenumber.
Note that a "discontinuity" (or near-) can be seen as a kind of Dirac pulse that is somewhat smoothed, and any method that has high spectral resolution capacity will be better in fact also in this case, only with some local filtering or damping because wavenumbers up to infinity can not be represented. Note also that being an advocate of low-order schemes is wholesale, because they are much easier, especially as for boundary and ultra-strong-gradients (Dirac pulse) treatment. But less accurate for sure for unsteady problems. A too simple stable solution is more a curse than a blessing. There is no free-of-labor healthy delicatess lunch.
I think that, in approximation methods, is not always possible to have accuracy increase, using more expansion elements. With high probability this is possible only by using orthogonal expansions and/or bandwidth limited function in the Sampling Nyquist technique. In Data fitting you can have high precision for some functions only, in despite of low precision for example in the derivative of the fitting or other connected function. Finally in Taylor expansion in the outside convergence zone, you need analytical continuation and so this is another limit. An example i have 200 terms in the Function error inverse expansion but only in a limited area. Regards. Giuseppe.
My old experience with high order spectral resolution compact schemes suggests that the answer is yes. In the case of shock waves the advantage is marginal since solution should be regularized anyway, but for convection-diffusion problems higher order was always better assuming that you take care about spectral image of numerical diffusion and dispersion. The best choice for DNS of turbulence. Beautiful resolution for vortex structures. But this rule works on very smooth grids. I know groups who are very excited with spectral elements (acoustics, EM, elasticity). Again the claim is that order increase improves the accuracy.
Dear
Vladimir A. Garanzha ,
thanks a lot, for sharing us your experiences!
- Badges
- Science topic
- Similar topics
- Mathematics