Dear Antoni, I strongly recommend Shogun (http://shogun-toolbox.org) , entirely written in C++, and one of the fastest libraries around. Its main focus is on SVMs, but it offers almost all machine learning functionalities you could require, including of course regression, plus it has bindings/interfaces for Python, R, matlab and many other well-known environments for data analysis. The only caveat is that it works best in Linux. If you're programming in a Windows environment, you'll need Cygwin, and then when I compiled it last time, I had to tweak the configuration files a bit to make it compile for my machine. Keep in mind it's not a small-sized library (the entire source code when I last downloaded it was about 100K LOC) but you can start using it in your project within a couple of hours after you start reading the documentation.

Thank you very much Ioanni!

I hadn't heard of Shogun before and it actually seems a very good toolbox and a quite active project.

I've once tried Shark http://image.diku.dk/shark/sphinx_pages/build/html/index.html

It's a machine learning library written in C++. It's major strength is its modularity which makes it easy to build your system letting you choose the model, the optimization scheme the error function and many other tweaks in an extremely easy way. I specifically used it for a classification task, but it offers plenty of methods for regression too. It's cross-platform thanks to cmake and based on the Boost library.