There are three layers of software involved in Deep Learning: general linear algebra libraries, tools to build custom network layers, and all-in-one packages that construct whole network given a few parameters. Higher layers are easier to use but offer less customisation.
For Python:
1st layer: Numpy, CUDA
2nd layer: Theano, Tensorflow
3rd layer: Caffe, Keras
I would suggest starting with Caffe/Keras, and only going deeper if you are not satisfied with provided functionality. All these packages are open-source (except that CUDA requires to buy expensive Nvidia graphics).
From Anton answer: "except that CUDA requires to buy expensive Nvidia graphics"
I don't think you need a very expensive graphics card nowadays to start with deep learning. An NVIDIA GTX 1050 TI with 4 GB for ~$150 or a GTX 1060 with 6 GB for ~$240 can be a good start to play with the deep learning frameworks+CUDA.
The usual comment is to buy NVIDIA TITAN X card for deep learning, but who the heck has X*$1200 in the pocket? :)