Is there a good open source symbolic Python package with automatic differentiation that can be used to write gradient-based optimization algorithms?
For example:
Sympy is a very nice symbolic package, however it uses symbolic differentiation instead of automatic, and the linear algebra packages i.e. matrix-vector and matrix-matrix multiplication are quite slow.
Autograd is an excellent option, with good quality code and some useful examples:
https://github.com/HIPS/autograd
You can try this : https://pypi.python.org/pypi/ad/1.3.2. Very good results if the size os the gradient is not excessively large.
Autograd is an excellent option, with good quality code and some useful examples:
https://github.com/HIPS/autograd
You have a full package for dynamical simulation and optimization here:
http://www.jmodelica.org
Take a look at the manual and look for Optimica. This is an extension of the Modelica language to formulate optimization problem. Then the algorithms in the package can apply various optimization techniques at your choice.
You, however, seems to want to develop new algorithms and this package is perhaps not really what you are asking for, but still could be interesting I think. I use it a lot myself for dynamical simulation.
/ Jan Peter Axelsson
Thanks Benjamin, this is exactly what I have been looking for.
I suggest trying TensorFlow.
Automatic differentiation in TensorFlow is robust and will work with almost all operations presented in TensorFlow and it will work through differential equations integrators, although, unfortunately there is currently only one integrator in TensorFlow ode45. TensorFlow is highly compatible and have much in common with NumPy.
TensorFlow graphs are fast and portable on GPUs.
But to use TensorFlow AD you will have to rewrite your functions using TensorFlow, of course, i.e. it will not work with your existing Python / NumPy functions. Although there is a mechanism to add your Python / NumPy functions into TensorFlow graph by using tf.py_func, but you will have to manually register gradients for your functions in order for AD to work on the whole graph.
More drawbacks of TensorFlow (TF) is that it is not so easy to debug your computations, you will have to use separate TF debugger.
And currently there is no capability in the core of TF to compute matrix derivatives such as Jacobians. I wrote a function myself to compute these.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python