From my personal view (which could be wrong), when we are young, the time that is passing (suppose one year) forms a huge part of your current age (like 10% if you are ten-year-old). So, you would feel that a year is a long time (relatively), and the opposite when we are older (a year is like 3.3% of your age when you are 30). I feel that the same analysis could go to memories, although the emotional state often affects the kind of memories that we think about.
Also from a personal experience, my emotional state became more stable as I grew up. Perhaps because I experienced many emotional states, and they have all become kind of familiar range (my mentality withstands the emotional roller coaster because I am familiar with many scenarios).
I hope this helps, and I would like to hear your take on it.