Just to clarify 'ex ante' returns refers to 'expected' returns as opposed to 'ex post' returns which are 'realized' or historical returns. Using the earnings estimates of analysts could be a way to estimate expected returns, but am not sure what the variance around that would be...?

The use of ex-post returns (or realized returns) as a proxy for ex-ante returns is due to the basic assumption of CAPM model. We assume an informational efficient stock market. Consequently returns should follow an stochastic process without memory (a random walk). According to this, the best estimator for tomorrow's prices is today's prices. In other words, the best estimator for future returns are past returns, which, in turn they cannot provide any forecast. You can read the book Foundations of Finance by Eugene Fama to clarify this issue. I hope this answer can help you with your research.

@ Bariviera : Your answer is the textbook justification for using realized returns as a theoretically valid proxy...thanks for stating that...however as we all know prices may not be mean reverting and often exhibit drift for prolonged periods leading to serious mispricings [recent Shiller Nobel prize on related themes]...it was in that context that I asked the community, what options we have. Cheers!

Almost all tests of the CAPM utilize ex post returns to test the model because observing ex ante expectations is difficult. Many of the arguments for why the empirical tests of the CAPM provide little support for the model involve arguments as to why the expected returns are not identical to the ex post returns. One paper, Brav, Lehavy, and Michaely (2005) utilizes expected returns and finds results consistent with the CAPM, supporting the idea that testing the CAPM with ex post returns is problematic.

@ W Johnson: Thanks for referring me to the 2005 paper, I'll check it out....my 1996 dissertation used dividend growth rates to develop the expected return vector via the Gordon DDM... so I really want to see the paper you mentioned...since expected returns on +beta stocks will be >0, the SML will always be positively sloping [in the expectational form] but as we know reality could be different and Rf becomes the best asset....but then people start saying CAPM does not work...whereas it is doing exactly what it is supposed to do... the Pettenglil, Sundaram, Mathur [JFQA 1995] calculates up/down betas and finds the CAPM is just fine....depends on the market phase.

You may also be able to get their actual dataset. I know for a while that Lehavy had it posted on his web site. If he has removed it, let me know. I am certain that I have it somewhere.

I have recently been paying more attention to empirically driven rather than theoretically driven predictors of returns. Robert Haugen has several papers that predict returns remarkably well (see for instance, Haugen and Baker JFE 1996). Of course, Haugen's point is that returns are predictable in a way that is inconsistent with risk (and inconsistent with market efficiency). But in theory, you could also generate expected returns using Haugen's ad hoc predictors of returns.

@WJ and others: as finance researchers we have to understand the difference b/n a explanatory model and a predictive model...the problem with finance data is that the MOST important data point is not yet in the dataset [yet to happen]...so forecasts will work as long as the mean-reverting process is in place, then suddenly hell will break loose [both on upside or downside]....

Try this code in Matlab, to generate charts that look very real but are the output of a random generator:

% file: Random Walk.m

% This file will create a Random Walk y(t)=y(t-1)+e(t)

% where e is a N(0,1)

% To run this file you will first need to specify

% T : number of observations

% starty : initial value

% The routine will generate a column vector

% and plot the vector

% Switches

T=100;

starty=0;

%Routine

y=starty*ones(1,T);

e=randn(1,T);

for i=1:(T-1),

y(i+1)=y(i)+e(i+1);

end

time=[1:1:T];

plot(time,y);

" the Pettenglil, Sundaram, Mathur [JFQA 1995] calculates up/down betas and finds the CAPM is just fine....depends on the market phase." Relating to testing CAPM with ex-post data: You will notice that betas in negative states are usually higher than corresponding betas in upstates. Call it inefficiency, CAPM imperfection, or mean-reversion bias...it is also referred to as "leverage" effect: in downstates, benefits from diversification tend to disappear since most securities' price fall. Interestingly enough some researchers have used this characteristic to demonstrate market inefficiency...and predictability--i.e., Avard, S.; Nam, K.; and Pyun, C. 2001. Asymmetric reverting behavior of short- horizon stock returns: An evidence of stock market overreaction. Journal of Banking and Finance 25: 807–824. You can look at some of my pubs too since I wrote on this topic.

Also, my 2cents on analysts' estimates and CAPM: In my humble opinion, analyst estimates (target/current-1 +div. yield or bE/P + g) are not fair proxies for expected returns. Capm's expected return is intertemporal while returns derived from analysts are period-specific (1Q or 1Y)...

@E Girard "You will notice that betas in negative states are usually higher than corresponding betas in upstates." This is interesting, I will have to look into it....Thx.

The basic problem with testing the original CAPM is that it is only a one-period model that users choose to extend to multi-periods for practical reasons. Another practical approach is to incorporate size and value effects into beta [as in Bornholt (2007) Accounting and Finance].

Hi,

I think you may find ideas in this paper:

C M Lee, E C So, "Uncovering expected returns: information in analyst coverage proxies"

C MC Lee, E C So Journal of Financial Economics 2017, forthcoming

Best regards

You can have ex-ante expected returns only if your market is efficient and participants have homogeneous expectations (at least you have to assume it). If so,  observed market portfolio (cap-weighted) would be "Sharpe portfolio",  i.e. it would one of many portfolios on unobserved efficiency frontier, and the only one maximizing expected return/expected risk ratio. 

So, in that case you can follow Black-Litterman logic (see e.g. Idzorek, 2004)  to reverse engineer market portfolio to obtain market expected returns for every asset in portfolio.

Historical returns as it is never should be taken as proxies for expected ones, it is known, like, for 30 years or so :)

I usually use the VaR and the Expected shortfall as a "ex-ante" returns and then procede to the estimation of the Black-Litterman model.