These are the minimal assumptions for these theorems. Moreover, differentiability in boundary points is always a little tricky, one usually defines differentials for inner points of a set. But one could demand that the functions are differentiable in a slightly bigger interval (a-eps, b+eps)
As stated, you have some nice examples to which you can apply mean value theorem, such as the square root function. This is not differentiable in the origin. If you want just Rolle's theorem, take sqrt{1-x^2}. The derivatives are not defined in 1 and -1, but Rolle's theorem can be applied on [-1,1].
This is how traditionally Theorems are formulated in Calculus. Of course, you would loose but little if you just require the derivative to be continuous even in a open interval containing [a,b], while students... most of them will not notice the difference. :)