dy/dx, zero derivative, no variation of y(zero), with variation of x, means the peak point(optimized point) of curve, where slop is zero.....

Since you have a function of several variables, therefore zero-derivative means all partial derivatives of the function are zeros. Certainly they will give idea of peak points(optimal) points but not necessarily . On the other hand, if a peak point is known in the interior then all its partial derivatives will be zeros. So peak points can characterize some of the zero-derivative points but not all.

Only meals that I can think of that does not use numerical or analytical gradients is simplex search method of Lagarius et al.

Formulating the problem as a fixed-point problem and solving it using the fixed-point iterations can be one way of finding stationary points of a piecewise smooth function without derivation.

In general, you need something in addition to differentiability in order to use computational approaches or other approaches that do not use differentiation.  For example, the total derivative (=gradient) of $f(x,y) = (1-x^2-y^2)^2$ vanishes at each point on the unit circle, and computational approaches in general do not produce uncountable sets of solutions.  The most common and a very powerful additional assumption is convexity, and given convexity there are many methods -- many which do not require a derivative.  However, simply being C1 over R^n is typically not enough.  

There are two questions here, one theoretical (knowing existence, uniqueness, etc. of critical points where the gradient vanishes) and one computational (finding such points).  The effectiveness of any computational method will depend strongly on structural assumptions giving answers to the theoretical question.  For even a local computation one must also be concerned with the character of the critical point -- a min or max is much easier to deal with than a saddle point. 

If one is seeking an isolated critical point with invertible Hessian, one can use difference expressions to get a computationally feasible approximation to the gradient and then setting this equal to 0 is a matter of solving a (probably nonlinear) system.  If one can start with a good approximation, then there are variants of Newton's Method for that which do not require an analytical form for its derivatives (the Hessian).   If one has convexity and coercivity, then one knows existence and uniqueness; the smoothness requirements become eased and there are several good methods for the computation. 

Summarizing, it is important to have some theoretical insight to guide the computation.

Thank you for responding to the question. Indeed, I  find each response to be enlightening and stimulating. Hence It appears that currently there is no  "standard" method for calculating zero-derivative points of C1 functions with a Lipschitz derivative which does not use the usual (tangential) derivative. However, in view of the "quadratic envelope" (QE) characterization of zero derivative points, let me rephrase, essentially the same question, as an equivalent geometric question:

Consider a compact convex set K with interior points in Rn. Consider a "nice" continuous NON-NEGATIVE function F(x,x*) such that F(x,x*) = 0 along x = x*  and also      P(x,x*) = c times norm(x-x*)^2, where c > 0 is some "large" fixed constant, both functions on the same product space. How can one find an interior point x* of K such that F(x,x*) is OVER-ESTIMATED by P(x,x*) for every x in K ?

(In the context of (QE) such x* is a zero-derivative point of f(x) and

F(x,x*) = abs (f(x) - f(x*).) Does this reformulation help answering the original question ?

Sanjo, I understand the spirit of your question, but, in general, optimization is involved and precarious, especially, for functions with multiple variables. With that said, depending upon the nature of the functions you are working on and with, I would consider Maximum Principle from Complex Analysis and Harmonic Theory. This may provide some means for identifying your maximas without differentiating the function. 

P.S. your paper, "On the behavior of functions ...", seems to rely heavily upon the idea of introducing a pole; hence, it might be even more advantageous to consider some techniques from Harmonic Theory to address these equations. 

Hello Luisiana,

You are absolutely  right ! My paper "On the behaviour of functions..."does rely heavily upon the idea of introducing some kind of a "pole into our context. This is another word for saying that a function      F(x) is "uniformly bounded" on K\{x*} if it is defined on K\{x*} and if there is a constant c = c(x*) such that  abs(F(x)) is bounded from above by c for all x in K\{x*}. I have used such functions also in  "L'Hopital's rule without derivatives" Mathematical Communications 17(2) (2012). (To find this paper on Google,  go to Mathematical-Communications-Hrcak-Srce and its archive).We know that for a  C1 function f(x) with a Lipschitz derivative, an interior point x* of a convex compact set K in its domain, is a ZERO-DERIVATIVE POINT of f IF,AND ONLY IF, x* is a "pole" of the function F(x) = F(x,x*) = abs(f(x) - f(x*))/norm(x-x*)^2  \ {x*} !

One could possibly call this statement, henceforth,  a "pole characterization" of zero-derivative point in contrast to the equivalent "quadratic envelope characterization" given in the preceding "answer". However, it bothers me that the term "pole" seems to come from complex analysis. I agree that it may be advantageous to consider some techniques from Harmonic Theory. Thanks.

Hello Sanjo, I am thinking of the Maximum modulus principle in particular, which states that for a function, for all z in the neighborhood of z*, if 

|f(z*)| >= |f(z)|, then the function f is a constant on the Domain, hence, it is a maxima.

So, to prove you have "found" the zero-derivative point sought, you would essentially prove the function a constant in the neighborhood of some critical point z*; thus, you would seek out where the functions become exceedingly flat... This may be done arithmetically, but you could also accomplish this by inspection, which is what I think you are driving for: a method that would not involve in-depth calculation, whether it be derivatives or else-wise. 

Actually, looking above at your definition of F(x,x*), I think we are saying the same thing.

Cheers!

One more point: once algebraically carrying out the terms, for at least polynomials, the leading term is the derivative, if you realize all other terms, minus one, equal zero near the neighborhood of x* -> 0. Finally, the one undesirable term that remains has a pole and that can be discarded for it does not contribute to the real part of the solution.

e.g. x^5 => ((x+x*)^5-x*^5)/x* = 5 x^4 +x^5/x* + 10 x^3 x* + 10 x^2 x*^2 + 5 x x*^3,

By inspection, in the limit of x* -> 0, all terms can be disregarded, except 5 x^4. The reason for seeking that limit is that is where the function is flat, with regard to the variable x*. Lastly, the pole contributes in the imaginary plane and can be disregarded for our purposes.

Although, this still is taking the derivative, essentially, but does remove a lot of the formality usually associated. 

As Luisiana and other followers  observed, our characterizations of zero-derivative points for C1 functions are closely related to various maximum principles. In particular, they give non-standard (not widely known) necessary conditions for local optimum. (Here I recall Fermat's extreme value theorem which says that f'(x*) = 0 at extreme point x*.) However, our results give GLOBAL descriptions of local optima and give it without using the derivative and solving f'(x*) = 0 !

Illustration: Consider f(x) = sin x. If x* is any local optimum then QE says that

abs(sin x - sin x*) is less than or equal to 1/2 (x - x*)^2 for EVERY x ! The dual characterization UB ("pole" characterization) says that abs(sin x - sin x*) / (x - x*)^2 Is uniformly  bounded on any compact interval I around x* for every  x in I\x*.

We also know that these results are sharper for C2 functions. The question is how to "extract" x* from these inequalities. Any idea ? Cheers !

These conditions are

Sanjo, that may be true, regarding the statement abs(sin x - sin x*) less than or equal to 1/2 (x-x*)^2, near a critical point; but, that is the problem attempting to algebraically solve for optimum...one cannot know the results for all Riemann sheets nor does your method determine the optimum point, in other words, you would a priori need to know the critical points in order to evaluate the optimum, but that's pointless, for you already know the placement of those critical points.

Hello Luisiana, Yes  - in the context of numerical optimization of C1 functions this may be the problem. Presently, in that context,  we can only verify  whether a given x* can possibly be  a local optimum. But other necessary optimality conditions, such as those of Fermat, Fritz John etc. do the same and, in addition, they also  require calculation of the derivative. Moreover, it seems that their conditions alone cannot be explicitly used to study sensitivity of the function around the known optimum, unlike the characterizations. Agree ? Have you seen the paper "Modelling with the twice continuously differentiable functions" mentioned on RG ? That paper gives some examples related to sensitivity that you may find interesting.

I'll certainly read your paper, the one I have read was well worth the read. I personally have had thoughts regarding the relief of functions, where I am coming more from a numerical analyst point of view. The Nelder–Mead method is such an example, but is wrought with troubles, especially, getting caught in a trough that is only a local maxima and not global. I personally think once you go beyond two-dimensional, the possibilities exceed our capabilities, except in the most stringent cases. 

On the other hand, analytic functions and representation of functions do give a rather general perspective of what can be represented and its likely behavior. Although, for the most part, these methods do rely heavily upon the idea and the ability to generate derivatives in the classic sense. 

I might say, the campaign you have embarked upon asks to accomplish a task that those in past times found derivatives indispensable; moreover, just because former investigators found such manipulations necessary in order to identify and study functional maximas, it is all together possible that a new set of tools could yield more fruitful results. 

I am aware of John M. Lee's series of books on Topology and differential manifolds...once again, most of the proofs, delineating anything of real interest regarding functions, rely heavily upon differentials. 

From Complex analysis, Goursat's lemma, seems to be directly applicable...

abs( ( f(z) - f(z*) )/ (z-z*) +- d ) < f'(z*), where d is a small number chosen and f'(z) is the derivative f(z).

So, for periodic functions, one would search within one period. For polynomials, the global maxima, depending on the interval, is either along the edge of the interval chosen or possibly somewhere in the interior, where that would be determined by the nature of the function, the interval chosen and algebraic inspection. 

To be honest, Sanjo, the derivative is quite frankly obvious. I haven't really noticed the obvious nature of it till this discussion of yours and forcing me to think about the nature of functions. Given the analytic representation, that is, a power series, then Goursat's lemma simply removes a zero before the coefficient leading the second power, hence, the coefficient for the derivative. All other coefficients would have some power, viz.: (z-z*)^n, {n|n in Integers (Z)\0 }

Also, given the confinement of representations available, that is to say, all functions are analytic within a domain and cannot be analytically continued beyond any singularity; hence, all are confined to some radius of convergence. Anyway, the long of it is it is quite obvious to me now. Honestly, no real need to ever calculate the derivative in the classic sense!

Great topic...