You can use Multiplot VMD and RMSDTT: RMSD Trajectory Tool plugin for trajectory analysis in VMD.

Plus, see the link:

http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/node7.html

When you use the formula for RMSD, instead of using the vector position r_i, use its projection along the direction you are interested (e.g., any direction n), so that in formula of rmsd, you replace r_i with

z_i = dot_product(r_i, n)