There are two approaches: economic model and statistical analysis. Below is the statistical approach. The economic model (CAMP) is attached in PDf file.

BENCHMARK

It is necessary to have a benchmark group, i.e. a group or an industry composite to use as a comparison against which the subject funs is evaluated. Obtain the descriptive statistic for this benchmark and use them as the theoretical value: mu = mean; sigma = standard deviation. The test statistic follows:

(1) Z = (x^ - mu) / sigma / sqrt n

… where Z = critical value of unit normal distribution; x^ = average yield of the mutual fund in question; mu = average yield of the benchmark group; sigma = standard deviation of the yield of the benchmark group; and n = number of sample fund.

For a confidence interval of 0.95, solve for x^:

(2) x^ = Z(sigma / sqrt n) + mu

The value for Z(0.05) = 1.65.

This is a preliminary test to see how the subject mutual fund stands vis-à-vis the benchmark.

T-TEST FOR MUTUAL FUND PERFORMANCE

Take the yield of the subject mutual fund over several periods and find the expected yield of the fund through the t-equation:

(3) t = (x^ - mu) / (S / sqrt n)

… where t = critical value for the fund yield; x^ = mean yield of the fund; mu = expected yield of the fund; S = sample standard deviation; and n = sample size: yield periods 1, 2, …n.

Solve for the expected yield of the mutual fund in question using 0.95 confidence interval:

(4) mu = t(S / sqrt n) – x^

The mu from t-equation is compared with the mu in Z equation. This comparison shows whether there is a significant difference.

PAIRED MEAN COMPARISON STUDY

Create two arrays of data: array1 = mutual fund denoted as A; and array2 = benchmark denoted as B. The benchmark may be an industry sector yield or market yield. Collect yield % for several period, i.e. 4 quarters in one year or 12 months average in one year and pair the arrays: (%:t) as show below:

Pairing Difference d(i)

A1: B1 A1 – B1 d1

A1: B2 A2 – B2 d2

… … …

An: Bn An – Bn dn

Now find the mean of the paired difference:

(5) d^ = (1/n) [d1+d2+…+dn]

The standard deviation follows:

(6) S(d) = Sqrt [(1/n) Sum (di – d^)^2]

The test statistic is given by:

(7) t(d) = d^ / [S(d) / sqrt n]

If the t-critical > 1.64, it means that the difference between the fund and the benchmark is significant. Under this scenario, if x^(fund) > mu(benchmark), it means the fund is doing better than the benchmark and this “doing better” is statistically significant. If the t-critical shows that it is less than 1.64, it is not significant; event if t(fund) > t(benchmark), they are still relatively give the same yield.

ATTACHED FILE

A file is attached file here for your reference. An article by Otten & Bams is a good introduction to CAMP and benchmark assessment of mutual fund performance. The suggestion above is statistical analysis, the article by Otten & Bams is economic analysis.

Thank you very much for the answer, I think it is very useful. I just have one question instead of the return for one mutual fund, can I use the average return of all the mutual funds that I am analyzing (or if I can say the average returns of the mutual funds industry)?

Yes, many mutual funds can be grouped so long as they are in the same industry. For example, there may be funds A, B, and C in the same industry. In this case, it would be a portfolio of funds; let's called it portfolio-1 (Port-1).

TAKING THE MEAN

Do not use the raw mean, i.e. sample mean = (fund 1 + fund 2 + fund 3) / 3. This approach may have sampling error. Instead take the mean through the standardize score under t-test.Use mu instead of x^ so that the researcher can set the level of confidence as high as 0.9999, 0.975 or 0.95. The mu determination follows:

(1) mu = t(S/ sqrt n) - x^

This would be the mean return of Port-1 at 0.9999, 0.975, or 0.95 confidence interval. That is, one can speak with say 95% confidence that the mean is mu = ?%

The performance evaluation is to compare Port-1 to a benchmark fund. Generally, in finance one uses T-bill rate as one of the benchmark components. However, you can construct your own benchmark model. For example, select the target industry that is withing the same industry of the fund and other industries outside of the fund's own industry for overall market comparison and run the comparison. The structure follows:

(i) Port-1 returns by itself

(ii) Port-1 against (T-bills + inflation)

(iii) Port-1 against own industry benchmark

(iv) Port-1 against whole market, i.e. NASDAQ or NYSE indices over same period

(v) Port-1 against GDP over same period

These 5 trends analysis by mean of comparison study would give a clearer picture of where Port-1's yield performance stand vis-a-vis its peers, the market, and the economy.

MARKOV PROCESS

Use the performance of the fund from prior period up to its nth term, i.e. t1, t2, ...,tn and predict the next period tn+1 via binomial probability answering the question: what is the probability that Port-1 will yield positive return or doing better in the next period(s)? Arrange the yield periods of Port-1 by time and answer Yes = success for doing better then its benchmark and No = fail to outperform or same as its benchmark. Score Yes = 1 and No =0.

t1 Y/N

t2 Y/N

... ...

tn Y/N

Find the probability of success by Laplace rule of succession:

(2) P(s) = (s + 1) / (N + 2)

Find the probability of failure:

(3) P(f) = 1 - P(s)

To find the probability of future period's successful performance X, use the following formula:

(4) P(X) = [n!/(n - x)!n!] [(p^x)(q^n-x)]

Note that P(S) = p and P(f) = q in binomial distribution.

The test statistics follows:

(5) Z = sqrt. [[(x/n) - p] / (pq/n)]

This type of test has a practical application in (i) let researcher predict yield performance in the following period in probability term and (ii) allowing the fund manager to use as a decision tool whether to change the fund's components assets if it under-performs. The above process may be used for about 2 subsequent periods and discount for reduction in accuracy, i.e. market stochastic drift.

My data set consists of 1580 U.S Mutual Funds with tot. assets equal or over 500 mil. So they are not industry specified, can I still apply the above mentioned methods?

Thanks a lot!

Yes, you can still use the approach explained above. Since the funds are not classified, you might want to classify the funds. The process below may help.

FUND POPULATION

In this case, N = 1580. This is a mixed population. In order to fully analyze it, the elements (each mutual fund) must be classified. Generally, the funds are classified into (i) industry, (ii) fund size, (iii) other classifiers. Once the funds have been classified, one can then pick specific fund groups or individual funds to compare against the yield rate of the population. The mean for the population is taken from the Z-equation:

(1) Z = (x^ - mu) / (sigma / sqrt n)

Solve for the group mean yield rate:

(2) mu = [Z(sigma / sqrt n)] – x^

The value of mu is the yield rate for the entire 1580 funds. This is the threshold value for evaluating other sub-group or individual elements. The group of 1580 may be used as a benchmark.

GROUP HOMOGENEITY

One may want to test whether the group is homogeneous, i.e. relatively having the same yield. One way to do it is to partition the fund into two halves: 1580/2 = 790 x 2. List these fund side-by-side as:

(x1:y1)

(x2:y2)

…

(x790:y790)

Now subtract each pair to find the difference of paired yield rates: d(i) where i = 1, 2, …, 790. Thus the pair difference follows:

x1 - y1 = d1

x2 – y2 = d2

…

x790 – y790 = d790

Then find the mean of d(i) which is d^ = (1/790) sum d(i). This is called d-bar. The standard deviation is given by:

(3) S(d) = sqrt [(1/n) sum (d(i) – d^)]

Homogeneity is determined by:

(4) t(d) = d^ / ZS(d) / sqrt n

The critical value for 0.95 confidence interval level is 1.65. If the t-observed is less than 1.65, it means that the group of 1580 is homogeneous, i.e. having the same yield rate. If the calculated t-observed value is larger than 1.65, it means that the funds have different yield. If that is the case, one should try to find out what contribute to the difference, thus, it is necessary to categorize 1580 funds into groups, i.e. industry, fund size, fund composition, investment philosophy, etc. After the classification, taking any classified sub-group from the main group of 1580 is called a sample fund.

SAMPLE FUND

In order to assess which fund is doing better, one can take the fund from a particular industry or from other name categories, such as fund size, investment philosophy, fund type (stock, bond, mixed, etc.). The sample fund with yield at periods t1, t2,k …, tn could now be analyzed for mean yield. The mean yield is given by the t-equation.

(5) t = (x^ - mu) / (S/sqrt n)

Solve for mu and compare this “mu” to the mu in Z-equation. This mu is the expected return for this sample fund. Now one can see whether this sample fund would outperform the group or perform under the group expectation.

The purpose of using the t-equation and Z-equation is to standardize the values and allow the researcher to speak with such-and-such percentage of confidence, i.e. 95% confidence.

OUTSIDE BENCHMARK

It may be necessary to construct an outside benchmark for investment yield. At least the following elements may be added as the starting components:

Rf = risk free rate

I = inflation

Df = default rate (if considered in debt instrument

O = other adverse rate affecting investment

There is a standard performance procedure used in research papers. For equity funds, this is the Carhart model; for bond and balanced funds, the FF93 model. Note that betas are time-varying and could also be augmented with time varying risk instruments.

For an equity fund, Carhart: RP(fund)=alpha+beta1(RPm)+beta2(SMB)+beta3(HML)+beta4(MOM)+e

For a bond fund or balanced fund, FF93:

RP(fund)=alpha+beta1(RPm)+beta2(SMB)+beta3(HML)+beta4(TERM)+beta5(DEF)+e

(See Fama and French, JFE 1993: Common risk factors in the returns of stocks and bonds).

Of course, you are interested in the alphas...

You can find the factors on French's website at http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html

I have 2 papers on fund performance you can look at: "SRF: goody two shoes or bad to the bone" and "FTSE islamic index performance"

I also want to add that the measurement of fund performance in itself is not as important as setting the research properly. Besides differentiating between closed-end and opened-end funds, you need to (1) look at "delisted" funds to avoid a "survivorship bias" in your performance analysis and (2) justify the time period of your study in order to control for a "period-specific" bias.

just sayin'

Thanks for the answer Eric, I am analyzing all funds grouped, so that I can give conclusions for the U.S. mutual fund industry as a whole, and the period I want to analyse is from 2002 to 2012 (monthly), which entails also the performance during the financial crisis.

Depending on if you want to trace out market timing or stock picking, you will find different measures. Stock picking can be captured by measures such as the Jensen's alpha, Sharpe Index, Treynor Index among others.

There are also some measures to track the fund manager's ability to anticipate market movements. A very simple one is based on Treynor and Mazui framework.

Personally, I think that you should split your sample into different buckets because your results will be highly sensitive to the choosen time span and benchmark.

Consider Performance Measurement without Benchmarks: An Examination of Mutual Fund Returns by Author(s): Mark Grinblatt and Sheridan Titman, The Journal of Business, Vol. 66, No. 1 (Jan., 1993), pp. 47-68 for an alternative method that is likely to give you more information about your sample and allows you to avoid the problems of finding the correct benchmark.

You can look at Data Envelopment Analysis (DEA) as an alternate method to evaluate relative performance

You can also consider a pragmatic perspective of doing comparative results over a set time frame. Or look it up on Morningstar. Bottom-line if you are a typical investor that is your best comparative source.

Now if you wish to work for Morningstar after your thesis is done, it might be worth making a cold call into several rating organization and ask what would they like to see assessed. That gets you to the leading edge of the research and they get free research. Either way it is a win for you because you will find a new analytical tool for the industry or rule out a research path. Both valuable and as employers they will be looking to assess you analytical skills sets.

This is a good topic, however there is a standard method for measuring performance in an organization, since you are measuring the fund performances. this could be measured using (DEA) Data Envelopment Analysis, which is non-parametric method for measuring performance. thanks

Do you have a hypothesis>?

You may wish to consider using cluster analysis to partition your funds into homogeneous groups and after that use some form of multi-factor regression analysis to estimate the fund alpha and the potential persistence of alpha over time.

I have a few papers that can be of interest to you.

Best regards

Mutual funds don't have alpha. They have negative alpha- unless you cherry pick the period studied, or fail to account for survivor bias.

You can analyze based on efficient market hypotheses

saibaba

Yes, Muthusway, and many institutions are allocating assets according to MPT. That's why index funds like spy, iwm, and lqd are very large. The interesting question to me is, at what boundary does efficient market theory break down? The efficient market requires that the market behave as though information is almost instantaneously transmitted. So, it's very hard to profit by selecting similar stocks. You can't profit by buying PG and shorting KMB, because the information that would justify a higher PE for one is already in the market. On the other hand, low beta and low P/B stocks have historically outperformed. I think it's unlikely the latter will do so in the future, because Fama's work has become so widely accepted. You could look at the recent past's significant outperformance of IWN as a reflection of the acceptance of Fama's ideas.

Move the boundary one more step. What about asset allocation? Are the prospects of each asset class the same on a risk adjusted basis compared to others at all times? What the stock market was half it's current value in 2009, which treasuries had zero or negative yields, were the two equally attractive?I didn't think so, and acted accordingly.

At some point, obvious market moving information becomes less important than judgement about the future course of events.

Hi

See the previous Literatures