I want to use Chi-Square test to compare the histogram of plain image and encrypted image. Please help me to understand how observed and expected frequency can be calculated.
your expected distribution will be the histogram of the plain image and the observed will be the encrypted one. It may not be important which way round you label them if you only need to know if they are different, not in which direction the difference lies.
I checked the paper and in this expected frequency is 256. But the suggested paper refers "Chaotic Image Encryption Design Using Tompkins-Paige Algorithm" paper, in which expected frequency is 64.
Chi square makes a hypothesis about your data and test the real values against it, if the result is good enough then your algorithm can be safe.
The hypothesis can be something like (in case of image encryption it is!) uniform distribution, so the encrypted image histogram should be uniform as long as it can or at least it should look like uniform distribution (with less error as possible).
Thus the point is: Making an assumption (which is called hypothesis here) about the data and test it to see if it fits or not.
To calculate the chi square itself, it is straight forward and i don't see any problem that anyone should have understanding it, the problems lies in understanding the steps itself.