A lot of debate takes place on what programming languages and their use when implementing numerical methods especially when it comes to problems that require the handling of large number of computations. Like in the case of FEM where professional programmers and scientists argue whether to use C, C++, Fortran, Matlab or other programming tools, in order to develop fast and accurate performing software applications. Do you believe that by using a fast language is the solution in developing a fast and reliable software application or the algorithmic implementation affects the most the final performance?
I think that the algorithm and its implementation are far more important than the programming language. Let me give an example of why I think that is. Some time ago I needed an implementation of a so-called cell tree. I got the implementation from a peer who already implemented it according to the paper. I wasn't satisfied with the initial performance of the implementation: It took 10 minutes to build the initial cell tree. So, I threw in some profiling to figure out the reason for this. First thing I did was replacing the STL vector (C++) with a QVector from the Qt library. Now, I was down to 3 minutes. Still not satisfied with the performance I tackled the implementation myself and improved some places where I thought there were some stupid choices. This brought me down to 1 minute. Finally, I got the original source code from the inventor of the cell tree. He had another trivial optimization which prevented multiple allocations/deallocations. Applying this trick to my source code brought me down to about 10 seconds.
This shows that not only the algorithm is important, but also its implementation. During the whole optimization the algorithm never changed, but its implementation did. So, the question is which programming language supports you in writing an efficient implementation of the algorithm. I've recently watched a video about Python for Fortran programmers. In that talk the solutions to the same problem written in both languages have been compared. Due to the lack of suitable libraries for Fortran the algorithm (and not only its implementation) was slower for Fortran than for Python. Not to mention that the Python code was a lot shorter. (For small input sizes Fortran was still faster.)
For an expert programmer, probably a computer scientist, the algorithm is far more important. Sometimes it is also important to choose between compiled and interpreted languages. However, for the average programmer the language is more important since it will define what algorithm you choose. As long as you use libraries with advanced data types it is easier to stumble on an efficient solution to your problem.
I also have to agree with Xinfu. From my experience I can tell that when you rewrite your code a second time it will always be cleaner and faster. Thus, using a scripting language for prototyping and a compiled language for the final implementation can have a great advantage.
Actually, there is some effect on the program performance, this effect occurs due to the behavior of the storage and retrieval variables which are used to solve the problem in memory.
If the application is small and if we use program like C, a decent written algorithm may not show as much difference against very well written program (algorithm wise). Given the same programming language, a well written algorithm may play a very significant role in the performance if the application is big. There is always optimization on the programming languages to choose from depending upon the applications (computing intensive not graphic, very graphical or both etc.) and ease of programming. Higher level languages like Java, Matlab and other take longer than C to execute. Like Zaid, said it also depends on processor technology, memory spaces, computer organization, materials used etc.
Hello, George! I think that your question is very interesting. Both are important. Before the algorithmic implementation is necessary choosing the programming language. The choice of which one to use is very important because this reflects on the computational platform to be used. After the choice of which programming language is going to be used, the algorithmic implementation also becomes too much important because this affects the computation performance. However, the several available tools in the literature makes it very easy to us. It is only necessary making the right choice, and I think that this depends on the problem to be solved.
I think it depends on what are your aime for
Personally, i have programed the FEM. using VB, Fortran, Matlab. my conclusion :
- If you are working on some personal program to test and validate some theory or contributions and you need an easy curves representation, all what you need is a clear and easy programing language as Matlab.
- If you are working on some advenced code, you need to use a scientific language "Fortran or C"
-If you are working on a professional code to be comercialised. You need advenced graphics, hardware acces, and anticrack protaction so, you have to use "C or java ...etc"
Both the programming language as well as the algorithm are important. But then, there is the third leg: the supporting libraries. If you can find good supporting libraries for you purpose, that can tip the balance because that reduces your burden a lot. With the libraries, you get access to some presumably optimized and oft-used code which will imply that it has also been well-debugged. I am also partial to open source code as that has been eyeballed a lot. If something is being used by your peers, using the same makes your code and algorithms amenable to peer review.
First off, the computational complexity of the algorithm is non-negotiable. O(n) > O(n2) in any language.
After that, your biggest issues are interfacing with people (use a language/environment that your developers are familiar with), interfacing with libraries (use a language that allows you to leverage other people's work), and interfacing with other systems (use a language/environment that makes integration easier).
For most development these days, the runtime speed of the language is less important than other considerations. This is especially true in an era where JIT compilers allow higher level languages like Java to run as fast as (or in a few cases faster than) a lower level language like C.
I think that the algorithm and its implementation are far more important than the programming language. Let me give an example of why I think that is. Some time ago I needed an implementation of a so-called cell tree. I got the implementation from a peer who already implemented it according to the paper. I wasn't satisfied with the initial performance of the implementation: It took 10 minutes to build the initial cell tree. So, I threw in some profiling to figure out the reason for this. First thing I did was replacing the STL vector (C++) with a QVector from the Qt library. Now, I was down to 3 minutes. Still not satisfied with the performance I tackled the implementation myself and improved some places where I thought there were some stupid choices. This brought me down to 1 minute. Finally, I got the original source code from the inventor of the cell tree. He had another trivial optimization which prevented multiple allocations/deallocations. Applying this trick to my source code brought me down to about 10 seconds.
This shows that not only the algorithm is important, but also its implementation. During the whole optimization the algorithm never changed, but its implementation did. So, the question is which programming language supports you in writing an efficient implementation of the algorithm. I've recently watched a video about Python for Fortran programmers. In that talk the solutions to the same problem written in both languages have been compared. Due to the lack of suitable libraries for Fortran the algorithm (and not only its implementation) was slower for Fortran than for Python. Not to mention that the Python code was a lot shorter. (For small input sizes Fortran was still faster.)
For an expert programmer, probably a computer scientist, the algorithm is far more important. Sometimes it is also important to choose between compiled and interpreted languages. However, for the average programmer the language is more important since it will define what algorithm you choose. As long as you use libraries with advanced data types it is easier to stumble on an efficient solution to your problem.
I also have to agree with Xinfu. From my experience I can tell that when you rewrite your code a second time it will always be cleaner and faster. Thus, using a scripting language for prototyping and a compiled language for the final implementation can have a great advantage.
Just my 2 cent
As programmer practically we usually know limitation or plus thing of our programming language. Thats why what is important for me to program is algorithmic, as its logical flow of program that we going to make. The implementation can be in any programming language as long as the algoritmic is reach and consistent, for further development of the program than we can go to optimization and its secuiryt.
I always say algorithm is the most important part than programming
Actually , Programming language is a tool for algorithmic implementation , based on this fact doing comparason between them is unaccurate
The only issue with the choice of programming language is whether it allows a reasonable expectation of an accurate implementation. The implementation may be possible in a language, but be too complicated to expect a human to create such an implementation.
For example, I have written reasonably complex subroutines in assembly language to obtain better performance, but I would write a complete Finite element code in assembly.
If the language has the capabilities you need, and allows enough higher level abstraction, then use whatever makes your job easier. I'm into performance, but if the implementation is not accurate, performance does not matter.
There is some discussion about MPI being too low-level, which has led to a move to the introduction of syntax which lends itself to distributed parallel operation being standardized in Fortran and hopefully soon C. HPF was not able to do this, but the co-array additions to Fortran in Fortran2008 show promise, as well as Unified Parallel C provided there is sufficient hardware support for the underlying operations, and support for adding UPC functionality to a C standard.
In addition to the excellent contributions above.
The algorithm is far more important than the language choice. However, implementing an intensive code on certain platforms such as matlab is just as bad as a very poor algorithm. So, the solution, is to develop an efficient algorithm and yet use a memory efficient programming language eg C++ and Fortran.
Even though am still at a very early stage of learning about algorithms and programming languages, i believe i have enough exposure to know for sure that the algorithm chosen affects the programming language of choice.
I believe writing the easiest algorithm for a particular problem determines how easy the implementation of the algorithm will be.
Some algorithms might require some data structures that are only available in some programming languages, the features of these programming languages also determine hoe frequently they are used in implementing algorithms.
I would like to thank you all for your comments and valuable feedback. I really appreciate your efforts and time that you invested in this question.
It is evident that the development of programming languages and especially compilers is rapid, providing the programmers with the ability to simplify their codes by using libraries and other means without significant loses in computational performance while in some cases achieving maximum processing performance.
From the discussion above and from my personal experience, I still believe that we live in an era where the programming skills are more important than the programming language itself, therefore the performance of the derived software and especially its extendability/maintenance depends on the algorithmic implementation, which are structured and developed by the programer.
After experiencing several times cases of two codes being written by different programmers by using the same programming language and built by using the same compiler, while deriving different performances and in some cases divergence of one of the two codes due to wrong algorithmic implementation issues, I believe that the implementation of an algorithm still remains more important than the selection of a different programming language that might perform better from the computational point of view due to an advanced compiler.
There is no doubt that in the future programming languages and their compilers will be able to provide algorithmic solutions that will require less programming effort without the loss of computational performance, but it is my belief that as the complexity in solving a physical problem through the use of numerical methods increases, the algorithmic implementation will always affect thus determine the accuracy of the algorithm therefore control its computational performance.
About 7 years ago at a conference in Santorini, Greece (COMPDYN 2007), was the first time that I heard a professor (I do not remember his name) stating: "who cares about the size of the numerical problem...you can buy a stronger computer and solve it...computers' computational strength increases all the time so the algorithmic implementation is not important". Since I started programming my own algorithmic implementations I discovered that actually the algorithmic implementation is not only significant but in some cases it will determine whether you will be able to numerically solve a problem or not.
Like the professor at that conference, currently there are programmers that might value the programming language and its available libraries ("...a compiler will come that will do for us...") more than the programming skills required to algorithmically implement a complicated numerical solution. Nevertheless, what makes the most important difference in this case when dealing with complicated algorithmic implementations, is the structure of the code and the selection of how the flow of the algorithm will be implemented so as to avoid memory leakages and achieve optimum algorithmic performance in terms of computations, accuracy and stability. Algorithmic implementation in by far more important when it comes to computationally demanding numerical problems given a certain level of solution complexity.
Who knows, maybe by adopting this approach (developing advanced programming skills) one might even find himself/herself in the position of being able to solve numerical problems with a personal laptop where others will require the use of advanced computer systems with increased memory and computational capabilities.
George, the quote "who cares about the size of the numerical problem...you can buy a stronger computer and solve it...computers' computational strength increases all the time so the algorithmic implementation is not important" that you've mentioned was quite common for a long time. And that statement was true for a long time because processors kept being faster and faster at a high pace (Moore's Law). But, the statement is not true anymore because even though processors keep improving in total performance, their single thread performance does not increase significantly anymore. And with parallel applications it is really important to design your algorithms correctly for high performance. There is no faster thing coming up that will improve your code's performance when the communication overhead is too high.
Because of your last paragraph I have an anecdote to share: At my department for several years a numerical code has been developed in C and no additional libraries were used. You needed a cluster to simulate your problems and it took a few hours. When that programmer left the department nobody really could maintain the code well. So, the decision was made to let a computer scientist rewrite the code (or rather the mathematical approach) in C++ using several libraries. After half a year we had a software that even included a graphical user interface and the code ran much faster. Instead of hours on a cluster it only took about 10 minutes on a low-performance laptop.
This a very interesting problems and I think the answer will depend on the the skills each interlocutor.
For me, for example, I think that it is important to find algorithms that to be deploy software as long as more optimally.
As someone answered earlier, computer science says the complexity of the algorithm is all that matters. One cannot take an algorithm with a polynomial complexity (like O(n**2)) and make it run faster than an algorithm with logarithmic complexity (O(log(n)); no matter how many different languages you try to write it in. The trick to making an algorithm run faster is to reduce its complexity; period. Once that's done, an algorithm with an O(n) complexity written in VB will ALWAYS run faster than one written in C/FORTRAN/... that has a (O(n**2)) complexity. So, the up-front investment/research in reducing complexity is the only answer that makes sense to your question. For instance, my computer science Masters thesis reduced the complexity of a string search algorithm from polynomial to linear and searched through the equivalent of 500 bibles for matches of dozens of strings like, "And the Lord said" of varying lengths in mere milliseconds. It happened to be written in C; but, any language would have achieved similar results. You see, I changed the algorithm into integer matching rather than character matching; thereby reducing its complexity. Implementing it was trivial after the complexity was reduced.
Dear Richard no one objects to the fact that by reducing the complexity of the algorithm will reduce the computational time but it is the algorithmic implementation in combination with the selected programming language/compiler that will eventually determine your overall performance.Especially when it comes to higher levels of algorithmic complexity.So stating that reducing the algorithm's complexity is the only answer to my question is not entirely true.If you take a look at my questions you will find another subject that refers to a very simple numerical example (do loops) that compares C++ performance with that of FORTRAN.There you will depict computational times of the same problem that derived by using the same compiler to built the same algorithmic implementation with some algorithmic modifications that led to entirely different computational performances.Even for such a simple problem the algorithmic implementation affected the final performance significantly.Especially when it comes to C++,even the way that you declare your variables affects the performance. Therefore it is evident that the algorithmic implementation is of great significance when it comes to implementing the solution of any numerical problem.
The question that I am referring too used the following code:
do i = 1, 100
do n = 1, 10000
do m = 1, 10000
iaA (n,m) = (n + 1) * (m + 1)
enddo; enddo; enddo
The required time for FORTRAN when using a 2,40GHz core was 4.8 seconds (30,000,000,000 calcs). Given that the reduction of the algorithmic complexity is not an option here, I will suggest to use the following code in VB:
do i = 1, 100
do n = 1, 10000
do m = 1, 10000
iaA (n,m) = (n + m)
enddo; enddo; enddo
This will result 10,000,000,000 calcs. It will be very constructive and enlightening if we knew the computational timings that VB will result for this numerically reduced problem.
Regards
There is no simple answer as there are very different applications. The time it will take to develop a program can be also considered. Some remarks :
-The choice of a language is also determined by the proficiency of the user in a given language. It affects a lot the time spent to develop a code.
-The time that you will spend enhancing your algorithm can also be considered.
-The complexity is not the only thing that matters as for small enough "n", a O(n) can be faster than a O(n**2).
-This article gives some information about the speed of languages http://economics.sas.upenn.edu/~jesusfv/comparison_languages.pdf
Of course, it is for a specific application and their implementations are perhaps not always optimal for all languages.
-External libraries are good to gain implementation time, but from my own experience they are not always fully optimized and obviously not optimized for YOUR application. For instance, the linear solvers of my own team was generally as much as fast as external solvers (for a same numerical method).
It is a very interesting discussion and I bring there my own experience.
Personally I think that both are important but not for the same reason. On top of this there is another ingredient to take into account is the development toolchain you have to use as it can have also an impact on the overall performance.
Algorithm is key regarding the way to solve the problem and it must address the issue of having the most suitable solution at solving the problem, and this in the constraints domain of the platform on which it will execute.
Then algorithm must address the nature of the problem regarding its complexity, accuracy and the nature of the execution platform regarding performance , scalability and more and more now: the energy efficiency.
For sure the algorithmic work is very important at the beginning, having an algorithm in O(log(n)) instead O(n**2) will more likely provide a better performance and accurate solution but unfortunately it is not enough. The algorithmic work is really the phase of transforming a mathematical/physical problem in a computer implementable solution but still a lot of job needs to be done.
The hardware platform on which algorithm will execute has an impact on the algorithm definition. Different algorithms have to be used depending on the hardware platform you are executing. Available resources are very different in an embedded computer, simple PC or super computer. It is as well true if you have only CPU or if very performant GPU are available. The implementation in each case can be very different and may be the algorithm itself can be different in order to fit with the available resources.
Regarding the programming language the constraints are different. Today I think that almost any algorithm can be coded in any language. For sure the performance and the time to do it can be very different form one language to another. Selecting a programming language is in first trigger by the availability of the language, and then the development environment on a given platform and secondly by the productivity which is offered to the programmer. For instance using a language having a rich computer vision library will certainly make implementation of a computer vision algorithm much more efficient than if the programmer has to rewite from scratch all the basic functions. But there is a lot of other aspects which need to be looked at when selecting a programming language than just the compiler one. All the aspects linked to debugging, profiling, ease of maintenance, portability form one platform to another are even more important than the compiler itself and they can have a very important impact on the programmed productivity.
So as a conclusion I consider that algorithm implementation, programming language and development environment are equally important and all of them must be selected very carefully. Even if some degree of freedom exist between these three pillars they are not completely independent and must but addressed somewhere concurrently in order to make sure that at the end the problem is well modelled( the right algorithm), meet the performance and feature requirements( the implemantation) in a reasonable development time and cost( the language and development environment).
All of us use computers to solve problems. So, we start with the problem. The next step is the mathematical representation of the problem, which is manifest in the equation set used to solve the problem. (Some problems may not have an equation set per se and may be able to go to the next step directly). Then the equation set is converted to a set of rules - algorithms - that can be solved on the computational hardware. Then comes the actual writing of the code which includes the selection of language. In this sequence, if you go astray in the beginning steps, you can end up with an untenable solution. For example if you decide to use an approximation of a process, you have to live with the errors inherent in it; they cannot be rectified by the choice of your programming language. Hence in my mind choice of the algorithm is far more important than the choice of the language. Of course, the choice of the language will be limited by whether you speak the language or not.
If you consider the language to be important, I would contend that you need to also consider the choice of the compiler into the mix.
@Richard: In general the complexity class is a good rule of thumb. But, the overall complexity (i.e. the constants in front of n2 for example) for a given input size is even more important. As an example, comparison-based sort algorithms are bound by O(n log(n)). But there are other algorithms that have a better complexity class. In fact, the sorting algorithm with the best complexity class that I have heard of (I can't remember its name) is faster than quicksort only for input sizes larger than 22024. That input size is never feasible. So, the algorithm and the complexity are important, but the complexity class is only half the truth.
The result depend on from your task understanding more than from language, or compiler (translator and linker). I you take the same algorith, realization of the algorithm depend as from your language knowedge as from compiler quality and used computer (processor and other hardware). Thus at least 3 levels of task solution can be looked. The first level (algorithm design) as rule the most effective, and it needed to other levels. If time solution no more, any languages are good. If solution time is visible the solution frequency give you the answer "programming language or algorithmic implementation?" is better. When you take the question, you need also add the question about hardware support. And in this case the assemler language may be give you the best practical result, if your task run often and have long runtime.
- Badges
- Science topic