Most common ways of comparing (sequential) algorithms:

1) Time-complexity

2) Space-complexity

3) Quality of solution (may not be applicable if it proposes to exactly solve a problem).  This one can be a huge factor if we're talking approximation algorithms.

4)  Ease of implementation/ adaptation of algorithm.  Some algorithms can have a lot of overhead to implement correctly.  There sometimes are more suitable algorithms for your application.  For example, in parallel computing, most will find Bellman-Ford more useful than Dijkstra's algorithm due to the way Bellman-Ford works.

These are of course things people that do research in Algorithms would consider.  If you're not interested in Theoretical Computer Science, then what Karthik may come useful (e.g., implementation's benchmarks and sensitivity).

Most common ways of comparing (sequential) algorithms:

1) Time-complexity

2) Space-complexity

3) Quality of solution (may not be applicable if it proposes to exactly solve a problem).  This one can be a huge factor if we're talking approximation algorithms.

4)  Ease of implementation/ adaptation of algorithm.  Some algorithms can have a lot of overhead to implement correctly.  There sometimes are more suitable algorithms for your application.  For example, in parallel computing, most will find Bellman-Ford more useful than Dijkstra's algorithm due to the way Bellman-Ford works.

These are of course things people that do research in Algorithms would consider.  If you're not interested in Theoretical Computer Science, then what Karthik may come useful (e.g., implementation's benchmarks and sensitivity).

Hi Vinita!

I would advise you to analyse the evolution of the objective function (OF) for each algorithm through "OF response" versus "number of evaluations" plots. This way you can easily compare the speed of convergence of the algorithms.

Then you can do the classical comparative tables containing: number of variables, constraints (or not), initial value of your OF, optimal value of the OF obtained, computational time, number of evaluations, (... and other possible information that you may consider relevant for the problems or benchmarks that you tested).

You can test your algorithms in distinct benchmark functions in case you want to verify if your algorithms perform well with distinct problems.

Good luck with your work!

Regards,

BC

To compare the performance of your proposed alg. with existing algorithm, you should focus on some performance evalution parameters such as time complexity, space complexity, detection rate, accuracy, sensitivity , precision and recall( in term of classification). You can also evaluate by input large data both of the algorithm and evaluate the result. 

If your algorithm performs better by some parameter then you can publish it.

Thanks all.

I want to do some statistical analysis on my data. please guide accordingly.  I have compared my algorithms graphically.

Thank

If you are comparing one algorithm to another based on some performance metric (i.e. Hypervolume Indicator) with a sufficient sample size, you may consider conducting a pairwise comparison using non-parametric tests.

A commonly used non-parametric hypothesis test is the Wilcoxon signed-rank test http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test

I have used this approach in my journal article: http://www.sciencedirect.com/science/article/pii/S0020025514010147

I hope this helps.

The idea to plot the speed of convergeance of your algorithm versus the old one is a good idea and you could also see if your algorithm could in practice put on multiples processors more easily than the old algorithm because this property could be a winning one in a world were the number of processors per computer is growing steadily and also see if it is scalable with the number of processors. If you could find theorically a formula for the speed of convergeance  of  your algorithm and compare with the formula for the existing one could be handy. The empiricals results should have significances but the formula for the speed of convergeance could be very convincing if you could also have the numbers of processors in the formula.

a. time complexity

b. space complexity

c. quality of solution that is produced by proposed algorithm

There are several ways to compare between algorithms and the chosen way is mainly dependent on your application. E.g. if you want to implement the algorithm in real-time/online, then computational complexity and time delay (latency) between algorithms may help to identify the most suitable (fast-enough) algorithm.

Another measure of course the traditional accuracy or error rate of the algorithm's output. 

In addition to the proposals above, a common method compare classifier algorithms is to compare the ROC curves.

Good luck!

To compare algorithms , we use generally terms of complexities in terms of execution  times and space memory. There are theorical and classical approaches.  Actually, there exist many applications domain and it's interesting to consider the adaptabilty of  new algorithm in the field  of application and the 'precision', reliabilty and simplicity of implementation. Finnaly, one can consider the 'cost' of the algorithm...

Yes, it is obvious that the most common ways of comparing (sequential) algorithms are Time/space complexity, quality and ease of implementation. It is also common to use other classic comparatives such as nodes, links, degree, etc. If you come up with new breakthrough complexity, it would be enough to compare with.  But if you did some improvement, it would be essential to compare performance.  

I agree with Gilbert. In general, you have to decide in a precise way what you mean by "better" and how maximizing that feature will provide something to those using the algorithm.

There are favorite obvious benchmarks, like how long it will take, or how much memory it will require. But in some circumstances, lower error (or greater accuracy) is the desirable feature. In others it can be ease of implementation. In others, just that it does something novel, for example allows a compiler to parallelize loops that other algorithms cannot.

All of those features can be probabilistic rather than deterministic; for example if you have a method that sorts numbers twice as fast *on average*, you have something worth people's attention.

It all begins with you defining "better" in a way that can be measured for both the old algorithm and the new, and in a way that third party colleagues  (like peer reviewers skilled in your art) will recognize (or upon reflection, agree) is a valid criterion for measuring the "performance" of this work.

Hi Vinita,

I believe it depends on what you are aiming for. If your goal is to imlement an already existing algorithm in a more efficient fashion (in this case, "better"=computationally more efficient), then you can compare the computational efficiency of your algorithm with those of alternative competing algorithms. If, however, you are aiming for a lower error (mean square error (MSE), for example), then your appropriate metric of comparison could be the MSE (in this case, "better"=MSE).

But, as you know, there is almost always a trade-off in science. What I mean is when you gain in one aspect, you generally lose in other aspects (similar to the well-known bias-variance trade-off in spectral estimation). Therefore, to make your work academically more sound, when you present the improvement of your algorithm based on your target metric (computational cost, MSE), you should also quantify at what cost this improvement is achieved  in terms of other metrics.

Depends on the type of algorithm. Comparisons are based on Time and space complexities basically. In some cases specific metrics need to be computed for measuring goodness of performance.

Already many suggestions have been proposed and all of them are important. I would propose one or two more points here. Provability of the algorithm is an important measure for comparison. Check if your algorithm is deterministic. An algorithm that is easier to understand is always better because augmentation / modification to suit a particular requirement can be made without difficulty.

Time and space complexity are vital and core. Prove the correctness of your algorithm. Find the benchmark to make your comparison even if you are comparing your algorithm against specific one. Try not to use weak variables in your comparison. Rely on strong findings. 

However, time and complexity are not *always* the desirable feature to maximize; I will provide two examples.

One is "numerical stability", a feature of algorithms in both linear algebra and in differential equations. We will expend both more algorithmic time and more algorithmic complexity on algorithms that are more numerically stable. In linear algebra instability is generally caused when an algorithm encounters numbers too close to zero or infiinity or both, because of the finite representation of numbers resolution can be lost. In general, this causes very slightly different numbers, just a few epsilon different from each other (where epsilon is the smallest number that can be added to one, on a machine, and get a result != 1), to produce large differences in the answer. So, for example, we expend a lot of time and space complexity by "pivoting" in the LU factorization in order to get a stable algorithm; without pivoting LU is not stable. Likewise, we have algorithms that do nothing but "condition" matrices in order to avoid instability.

A second example is "robustness" in its technical statistical sense. Robustness means a measure or algorithm that is *not* sensitive to outliers. So, for example, when I fit a least squares line to a set of points, the line can be heavily skewed by a single extremely large value. Even if I have 99 pairs of numbers that form  a perfect 45 degree line, if I add a single outlier pair for which the Y value is hundred times what it "should" be to fall on the same line, the fitted line will be severely skewed off the 45 degree line. The algorithm will "give up" the perfect fit for 99 points in order to better represent the one outlier point. That is *not* robust.

A robust least squares approach is, instead of finding the line that minimizes the sum of squared errors created by using that line, to find the line that minimizes the *median* of the squared errors created by using that line. In the example, that line is the 45 degree line, and the median of the squared errors is zero, and the outlier point (and indeed up to 98 more crazy outliers) would be ignored completely. The 45 degree line would be found.

Similar robust algorithms exist for fitting statistical curves, finding shapes, etc. However, they are costly in time, sometimes severely so. But the time cost is worth it because robust statistics are effectively immune to "noise" in signals, where least squared error approaches are not. The real world is full of both noise and outliers, some of them caused by human or machine error, in particular in pixelated images or sampled sound. In such circumstances, a more time-costly robust algorithm can find signals in a noisy environment where faster algorithms produce answers so corrupted by noise that they are useless, or even dangerous if accepted.

In some cases, time and space complexity are not nearly as important as finding an answer most likely to model the actual signal despite noise and degradation of the signal.

Dear Anthony,

I do agree with you that numerical stability and robustness are two very important criteria for an algorithm.  

In general i think the algorithm which is simple giving solution with lower overheads is the better one. Also it should follow time and space constraints.

Thanks all.