You need hyperspecteral camera, and capture photo and spectrum gives you information about calories. If you have only 2d images captured by CCD cameras, only can use RGB, YCBCr and HSV values. Previously, you must get data about calories (measuring NIR, Intensity or chemical analysis), after, you take RGB, YCbCr and HSV values. Using artifical neural network or different statistical methods, these methods learn to use your data, after that, you estimate the calories.
Maybe you have to use object detection on food images, check out which materials were used in food. Then calculate the total calories value by summing each material' portion(in food) * Calory value.