I've heard many perspectives, and "textbook answers", but I wanted to know if anyone is experienced with languages, like LISPs who preferentially using functional-programming based principles for AI.
Programming language debates become heated very quickly. My advice should be taken as the personal opinion of a single researcher (although with many years of programming experience in academic and commercial projects). First of all, I suggest to
a) use what you are comfortable with (a project done in Visual Basic is still better than no finished project at all), and
b) learn a new language every now and then, to expand your mind and programming style
Personally, I would apply the following criteria for choosing a language for AI projects:
- Clean and concise expression of my thoughts. (I dislike the boiler plate of C++ and Java.)
- Mature toolchain: I want testing suites, debugger, code navigation etc.
- Available libraries: I do not want to re-write UI frameworks, machine learning libraries etc from scratch
- Rapid prototyping: my code should be fluid, adaptable and maintainable: AI projects focus on novel algorithms, not on codebase stability and team scalability. (Python fits these criteria better than Java, for instance.)
- Support for specific features of my project: if I need CUDA or large scale concurrency, I either need a language with support for these things, or a way of interfacing cleanly.
Since AI projects are mostly academic, you might want to focus on how well others can understand your code.
Now about functional programming. I think that the major benefits are:
- Closer alignment to mathematical thinking; you tend to spell out your ideas in a format close to mathematical definitions rather than iterative programs.
- No (or at least fewer) side-effects: your functions do not influence other functions, which is great for concurrency and parallelization, but also for debugging.
- Fewer lines of code, without sacrificing conceptual clarity: Lisp is more powerful than non-functional languages, in a similar way as C++ is more powerful than Assembler. You spend a greater proportion of your project with thinking than writing, but eventually, you will probably find that you are more productive (get more features done in less time). Also, fewer LOC means easier maintenance.
Some of the features of Lisp have made it into non-functional languages (functions as first-class objects, closures, list-comprehensions etc.). Python might really be a good choice if you want to get familiar. But the power to redefine the language itself cannot easily be transplanted into languages with less fluid syntax.
Programming language debates become heated very quickly. My advice should be taken as the personal opinion of a single researcher (although with many years of programming experience in academic and commercial projects). First of all, I suggest to
a) use what you are comfortable with (a project done in Visual Basic is still better than no finished project at all), and
b) learn a new language every now and then, to expand your mind and programming style
Personally, I would apply the following criteria for choosing a language for AI projects:
- Clean and concise expression of my thoughts. (I dislike the boiler plate of C++ and Java.)
- Mature toolchain: I want testing suites, debugger, code navigation etc.
- Available libraries: I do not want to re-write UI frameworks, machine learning libraries etc from scratch
- Rapid prototyping: my code should be fluid, adaptable and maintainable: AI projects focus on novel algorithms, not on codebase stability and team scalability. (Python fits these criteria better than Java, for instance.)
- Support for specific features of my project: if I need CUDA or large scale concurrency, I either need a language with support for these things, or a way of interfacing cleanly.
Since AI projects are mostly academic, you might want to focus on how well others can understand your code.
Now about functional programming. I think that the major benefits are:
- Closer alignment to mathematical thinking; you tend to spell out your ideas in a format close to mathematical definitions rather than iterative programs.
- No (or at least fewer) side-effects: your functions do not influence other functions, which is great for concurrency and parallelization, but also for debugging.
- Fewer lines of code, without sacrificing conceptual clarity: Lisp is more powerful than non-functional languages, in a similar way as C++ is more powerful than Assembler. You spend a greater proportion of your project with thinking than writing, but eventually, you will probably find that you are more productive (get more features done in less time). Also, fewer LOC means easier maintenance.
Some of the features of Lisp have made it into non-functional languages (functions as first-class objects, closures, list-comprehensions etc.). Python might really be a good choice if you want to get familiar. But the power to redefine the language itself cannot easily be transplanted into languages with less fluid syntax.
I find functional programming good because it is mathematical and thus open to simplification. In some cases it can be better to have a less simple more verbose code but maybe that is what comments are for.
I have some experiences on Ocaml, a commonly-used functional programming language in building compilers. I believe functional languages like Ocaml is security and good at text processing.
Well, it depends what are the objectives. If one wants to develop small applications with time requirements, then languages such as Lisp or Scheme is a good choice. We are working also on a language based on Java encapsulating Lisp (see e.g., learn-clojure.com/clojure_tutorials.html).
Alternatively, if the main objective is to focus on the logical aspects, Type Theory with languages such as Coq or Haskell is a better choice since it is more expressive than classical Prolog-like languages while offering well-established theorem provers (see e.g., www.math.ucsd.edu/~mshulman/hottseminar2012/02typetheory.pdf). Notice that Coq is built in OCaml.
I think that might also be historical coincidence. The originators of LISP were involved in AI, and there was some great hope for AI in those days -- and they wanted to have a more declarative flavor in programming the systems.
I agree with Joscha's appraisal, and would just add that some of the features of LISP which have been useful in Artificial Intelligence and Natural Language research make it easier to explore compositionality in complex data (e.g. in language, music, theorem proving); Python shares some of these properties. But the many practical issues which Joscha mentions also need to be considered.
I am wondering whether someone has experience in using the Jason framework (http://jason.sourceforge.net/wp/). It should be used for the development of cognitive agents in BDI (belief, desire, intention) paradigm. This is strongly concerned with the AI field.
I agree with Sandeep - it is a remnant of software history. Lisp was promoted as an AI language for shallow reasons, but in a new and fertile field, the idea took hold. The idea reached its almost comic zenith in the '80s when, in response to the Japanese 5th generation project, the U.S. gov't and industry poured money into the creation of Lisp machines. Someone forgot to tell them that running dumb software faster would not make it smarter.
I'm not anti-Lisp, I just think its special applicability to AI is and always was an illusion.
Yes, the interpreted LISP (and Prolog, prominent in Europe from the 1970s on) are basically part of software history now; the LISP machine produced fascinating results at the time, but was rapidly overtaken by hardware developments. Many useful features, such as easy recursive search or garbage collection, have long been standard in other languages.
To be more explicit: LISP and its variant Scheme have been used for many purposes (cf. emacs, DSSSL). The reasons for using LISP in AI were anything but shallow or dumb: logic, reasoning, theorem-proving were key interests of early AI, and the clear semantics of large subsets of these languages supported this priority. For such purposes in prototyping they still have their merits, as do other interpreted languages such as Python which have largely replaced them. Interests have changed. For applications, more efficient compiled languages with modern sophisticated development environments are called for.
So to answer Jon's query more directly: it depends very much what AI topics and goals you are interested in. And, yes, functional programming is useful for many purposes, not only teaching. You may find it useful to check out the Scheme variant of LISP.
I am an Ex Symbolics Lisp Machine (1980s) user. It was (and still would be if they still were around ) one of the best rapid prototyping systems ever. Languages such as python and C++/java/c# have acquired some of their facilities, but still none are as good as Flavors and Genera. If you use an ordinary Lisp language such as scheme you unlikely to get the same environment as a Symbolics Genera - its just another language to learn.
Although we used our machines for AI studies - its not the language that makes the difference to the AI but the facilities and environment. Some of these facilities are available with with current languages (Object orientation for example). AI is not really about computing - we use to computers as the modelling tool for AI.
Use whatever language you feel most comfortable with.
I would like to ask you back, what is basically common to Lisp and all other computer languages and AI? My answer is nothing! You can find some effective algorithm for special programming a robot for industry and for cleaning my apartment, but it has nothing to do with something that came one and only one step near to human brain function. Google research is trying to compile knowledge about people to a program that understand semantics of human language. This giant plan is also nothing else than a chain of sophisticated algorithm that give answers only on based pre programmed commands. I can only repeat to say first you must have a logical structure with adequate structure to brain function, than you get operational logical values and only than you can think about something like AI
I agree the languages are nothing - re-read my earlier comment.
I think that one must understand that biological systems are innately heuristic structures and that logic is a restriction of heuristics that humans use in association with some form of language (maths included).
I used my lisp machines to build simulation models of real-time systems (non-deterministic) and their environment. This leads to a state-based model of control - with only a modicum of intelligence. It was the consistency of the environment and quality of feedback from that environment that was significant
If you need to go for production speeds and like a functional programming language python is quite good. If performance can be a problem as you can get a complied version using IronPython (on windows machines or at least with .net) or Jython for compiling to java bytecode. IronPython can be used as production code (I do not have experience with production stability with Jython).
I agree with Joscha Back that chosing the right tool for the job is worth spending a little time on before you start. External libraries can spare you from a lot of implementation but they can also be more hazzle than it is worth if they are of a different architecture than you want to use.
In my experience the most problematic is to choose a programming language/paradigm that the programmers does not know. If they are very good in Java or C# you can do almost as much in those languages as you can in functional programming languages without the performance degrading much because they have extensions making it possible to program functional when you have to.
LISP has an advantage if you want to develop symbolic representations. You can simply write them directly as LISP lists. In particular, the very powerful macro functionality of LISP (which unfortunately is missing in Python) means that you can write problem descriptions in your representation language directly as LISP code. You can very easily define your own language (as long as you adhere to the parenthesis syntax).
Some people do say the equivalent of "language is nothing" or "it does not really matter".
This is not true. Language guides thought, and thought guides language.
In particular, different programming languages make some things easier and others more difficult.
FP is good for AI because FP languages tend to be problem oriented; whereas languages like C and Java tend to representation oriented (they pay a lot of attention to the representation of data).
Modern FP languages bring additional benefits - a very rich sense of control abstraction for example.
Why is functional programming advantageous in Artificial Intelligence?
For my opinion, the knowledge of pure functional programming does not give you any advantage in solving problems.
Let's start from the beginning:
First, you have a problem that needs to be solved.
Second, you need let say methods of Artificial Intelligence (known or unknown), to solve your problem.
3rd, in literature or on the web, try to find out the methods of AI appropriate for your problem, or make (think) it up.
4th, apply the solution to solve your problem.
5th, you've got a solution, then you know the answers to questions: what, how, why, etc.
6th, it remains to apply the solution on your PC, i.e. to make an appropriate program.
Functional programming or better the knowledge of functional programming paradigm (and/or problem solving) could help you in faster solving your problem. But, the knowledge of logic, object-oriented, imperative, declarative, event-driven, etc. programming (paradigm) could help, also.
I've made (or be involved) in design and realization of some expert systems, intelligent tutors, intelligent agents, and the like. Never, in solving such problems, felt that knowledge say LISP or PROLOG, gives me an advantage. More important is a good mathematical, psychological, cognitive and similar base (theoretical foundation). Then the programming language or programming paradigm you use is less important.
What's AI? As the promise of 60s - 80s seems to be waning (as some people have already indicated here), the flourishing Machine Learning is sometimes considered a part of AI field.
I have ancillary experience with functional programming, so please correct me if I am wrong: as the project matures and efficiency becomes an issue, most of the conceptual clarity goes down the drain
From 1957 to about 1995, AI R&D was based largely on symbol-centric models and techniques. Lisp was good at implementing these because Lisp is lists, and lists are perhaps the most flexible way to represent aggregates of symbols. In Lisp, lists, trees, graphs, etc can be manipulated as first order programming constructs (esp. using its OOP extensions and macros). However this capability had little to do with functional programming principles as seen in other FPs like Haskell, ML, Erlang, etc. In fact, other FP languages were never widely adopted by the AI community, nor were FP theory principles in general.
Since 1995, AI models and techniques are no longer as symbol-centric; they place much more emphasis on engineering-based numerical solutions based on probability, optimization, decision theory, and pattern recognition. Lisp and other FP languages (with a few exceptions, like Scala or F#) do not excel at such numerical tasks or they lack the extended support libraries needed (like numpy or scipy, for example). This makes them less competitive than Python or C++.
One FP language that *is* used a lot in AI-ish applications is R, which has excellent support for statistical discrimination/description of data (classification, clustering, etc). IMHO R's strength lies less in its FP attributes than in its extensive suite of packages that provide a wealth of sophisticated statistical tools. Likewise, I would expect any stats-centric language with a comparable set of extensions should perform equally as well as R toward serving those ends, regardless of whether it's a FP.
The one you are the most proficient with! There are a few exceptions though, e.g. if you want to artificially evolve a program, it will make your life easier to write it in an interpreted language.
Here is a more precise answer from comp.ai written in 2004, which is worth reading thoroughly once for all:
There is no authoritative answer for this question, as it really depends on what languages you like programming in. AI programs have been written in just about every language ever created. The most common seem to be Lisp, Prolog, C/C++, recently Java, and even more recently, Python.
LISP: For many years, AI was done as research in universities and laboratories, thus fast prototyping was favored over fast execution. This is one reason why AI has favored high-level languages such as Lisp. This tradition means that current AI Lisp programmers can draw on many resources from the community. Features of the language that are good for AI programming include: garbage collection, dynamic typing, functions as data, uniform syntax, interactive environment, and extensibility. Read Paul Graham's essay, "Beating the Averages" for a discussion of some serious advantages: http://www.paulgraham.com/avg.html
PROLOG: This language wins 'cool idea' competition. It wasn't until the 70s that people began to realize that a set of logical statements plus a general theorem prover could make up a program. Prolog combines the high-level and traditional advantages of Lisp with a built-in unifier, which is particularly useful in AI. Prolog seems to be good for problems in which logic is intimately involved, or whose solutions have a succinct logical characterization. Its major drawback (IMHO) is that it's hard to learn.
C/C++: The speed demon of the bunch, C/C++ is mostly used when the program is simple, and execution speed is the most important. Statistical AI techniques such as neural networks are common examples of this. Backpropagation is only a couple of pages of C/C++ code, and needs every ounce of speed that the programmer can muster.
Java: The newcomer, Java uses several ideas from Lisp, most notably garbage collection. Its portability makes it desirable for just about any application, and it has a decent set of built in types. Java is still not as high-level as Lisp or Prolog, and not as fast as C, making it best when portability is paramount.
Python: This language does not have widespread acceptance yet, but several people have suggested to me that it might end up passing Java soon. Apparently the new edition of the Russell-Norvig textbook will include Python source as well as Lisp. According to Peter Norvig, "Python can be seen as either a practical (better libraries) version of Scheme, or as a cleaned-up (no $@&%) version of Perl." For more information, especially on how Python compares to Lisp, go to http://norvig.com/python-lisp.html
This topic has been debated many times before; and there are lots of reviews on it across the web. It depends on what your building blocks would be. LISP allows a natural means to represent tree like structures which can be used to represent building instructions, mathematical functions etc. That was why genetic programming first used LISP as stated by Koza (1992). This however does not mean that you cannot implement such approaches in other programming languages. It's a matter of personal preference in my opinion considering the available libraries... For instance, for me learning/reading/implementing C# code was easier than C++
I've been working in the ALife/AI area for a few years, and I use Haskell, a functional programming language. But my reason for using Haskell has nothing to do with it being especially suited for AI. I use it because it allows me to produce high-quality, bug-free code faster and more effortlessly than any of the other languages that I've used. Some aspects of functional programming make it easier for the compiler to catch bugs, and make it much easier to test your code thoroughly.
If you've only worked with procedural languages, learning a functional language is useful because it gives you another way to think about problems. However, it takes significant time to learn the functional way of doing things, which is something to consider if you're on a tight schedule.
As Franck said, if you're evolving code, then an interpreted language has advantages. Also consider what libraries are available that you might want to use. But ultimately, any general-purpose language can do the job, so you can use what you are comfortable with and enjoy using.
I think the language selection is highly problem dependent and sometimes you have to learn a new language just to solve a particular problem if you can't find someone who can make the program for you.
LISP is a classic, used in AI for many decades, For some problems PROLOG works very well. I've used it to make expert systems before the development of specialized shells and also to obtain derivatives and expand expressions in Taylor series before mathematica.
Sometimes you require speed, like in data mining, and there the best is to choose a language that works as close to the operating system and hardware as possible. In this case C is a good choice. Interpreted languages like Java or Python will take forever solving some problems. Recently, working in a DNA assembly problem I started using PERL (similar to Python) but it was very slow. Then I reprogrammed the same routine in C, with some very minor changes and could obtain the answer about 800 faster.
With time and experience you will have learned many languages and the choice of the appropriate language will be much easier.
All computer languages, even those that are interpreted, map onto a relatively standard hardware architectures. That implies there is a mapping from language to hardware and using that, all problems could be seen in terms of the hardware.
Thus it can only be the linguistic form and its ease of understanding that matters. This is equivalent to saying: use the language you are most familiar or happiest with. You will find many replies from different questions which say this - often from more experience practioners.
AI is not about computers. Computers may be used in simulations of AI models. For simulations any language will do. For AI only, you need to do something different. Try building new hardware along the lines of sensors and control systems first, then make them adaptable (with respect to a consistent environment) , then start to add internalise feedback to close those open feedback loops.
The topic of different languages has been answered whether they are logical, functional or imperative - it doesn't matter. If its AI you want then, we need a change of philosophy or perhaps basis.
In my country people say: “Thousands people, thousands opinions!” For people involved in AI, especially in “the best and only programming language in AI” we could say “Thousands people, billions opinions!” So, my advice is: “Follow your heart, and your research goal, and work hard.” Then it does not matter whether the functional programming is "the best" for AI! No shortcut is allowed.