Usually we have two versions of cepstrum: complex and real cepstrum which depend on whether we take logarithm of FFT or the absolute value of FFT. Suppose a real signal x(n) of length N and a real FIR filter h(n) of length M, N>>M.
Question 1: the complex cepstrum is also real, i.e., ifft(ln(fft(h(n),M))) is real. How to prove this?
Question 2: I have verified using Matlab that if I change the FFT point, say calculating ifft(ln(fft(h(n),2*M))), then the results become complex. How to explain this?
Question 3: According to the above two questions, we come to the classical application of separating signal and system convolution into addition. Let C denote cepstrum, the system output is y(n)=filter(h(n), 1, x(n)) which leads to Cy=Cx+Ch. In all literature I reviewed, it is implied that Cy is real, however, to obtain Cy we need to calculate fft(y(n), N)=fft(x(n),N)*fft(h(n),N). Note that ifft(ln(fft(h(n),N))) would never get real values because of Question 2.
How the above are actually calculated?