FDM is an older method than FEM that requires less computational power but is also less accurate in some cases where higher-order accuracy is required. FEM permit to get a higher order of accuracy, but requires more computational power and is also more exigent on the quality of the mesh.
Also, you find at Article On the Comparison between Compact Finite Difference and Pseu...