Hi there,

Theano does support 3D convolutions, so you should not have trouble implementing 3D CNNs. There is a lightweight library called Lasagne ( https://lasagne.readthedocs.io/en/latest/ ) that is built on top of Theano that you can use to build neural networks, and that supports 3D convolutions ( http://lasagne.readthedocs.io/en/latest/modules/layers/conv.html#lasagne.layers.Conv3DLayer ). I use Lasagne myself, since it makes common operations easy to define, while still allowing you to access lower level Theano features easily if you need them.

For R-CNNs, it depends. For the earliest version of R-CNNs, you should be able to get by with the available operations in the libraries (since R-CNN is based on normal CNN operations). For fast/faster R-CNN, it is more complicated.

For fast/faster R-CNN, there's a discussion at https://github.com/Lasagne/Recipes/issues/35 about implementing these using Lasagne / Theano. One of the commenters in the thread has suggested changes to Theano and Lasagne, although at time of writing those changes seem not to have been merged into the main branches of the two libraries. If you need to implement these in a 3D context (this may or may not be the case depending on what you intended in your question), you will likely have to make changes to the code also.

Unfortunately, there is not really a way to combine models in Theano and TensorFlow into a single, joint-trainable model easily. But, it might be worth mentioning the Keras library ( https://keras.io/ ). This library performs a similar function to Lasagne, it is used to quickly and easily define neural networks. It is different from Lasagne in that it allows you to use either Theano or TensorFlow as backends. You still have to choose one or the other though. Keras is popular, and is a reasonable choice if you want to keep using your TensorFlow skills. However, because it tries to abstract both Theano and TensorFlow, it can't be as close to Theano as Lasagne is. So, access to Theano's internals isn't quite as simple and direct as for Lasagne. But, I don't personally use Keras, other commenters might want to chime in. I do notice that there seems to be an implementation of Faster R-CNN available on GitHub ( https://github.com/yhenon/keras-frcnn ).

So, to summarise, 3D convolutions shouldn't be a problem. They are supported in Theano, Lasagne and Keras without any additional work, you just have to define your CNN using 3D operations instead of 2D ones. Apart from basic R-CNNs, things seem more complicated for fast/faster R-CNN. At the time of writing, there is code available out there, but this seems not to have been incorporated into the main codebases of Theano, Lasagne or Keras. If you want to apply fast/faster R-CNN in a 3D context (if this is what you'd like), you will likely have to modify this code.