Shrinkage of covariance matrix

Posted On: 2014-11-15

A common pitfall in portfolio optimization is error maximization, which can be caused by an insufficient number of samples, noise and many other factors. One of the main issues is to estimate the covariance matrix. It's trivial to find the sample covariance matrix, but with a limited number of samples we do not know how well the sample covariance matrix represents the actual covariance matrix.

If the sample covariance matrix suffers from estimation errors, this will perturb the mean-variance optimizer. What happens then is that extreme values, which tend to suffer most from estimation errors, take on more extreme weight values from the optimization as they will have a larger effect on the objective function or constraints. This problem is more obvious when the sample period is less than the order of assets in the portfolio.

Frankfurter, Phillips and Seagle (1971) conclude that portfolios selected based on the Markowitz criterion rarely outperform or are no more efficient than an equally weighted portfolio due to sampling error being too large.

Jobson and Korkie (1980) find that when samples are small the sample estimators tend to be significantly different from the true ones. When the sample size is at least 300 they tend be reasonably comparable. If the asset returns however are small in relation to the off-diagonal elements in the covariance matrix, this number will be significantly higher. The conclusion is that the mean and covariance matrix do not lend themselves to making interferences in small samples, which are often the case in portfolio optimization. A more robust estimator is therefore highly desired.

There are many approaches to deal with this problem. Industry standard is to use multi-factor models to allow more flexibility and to reduce the bias in estimation, but often lead to an increase in error maximization. An alternative approach is to use shrinkage, where the sample covariance is shrunk towards the real covariance matrix to decrease the risk of estimation errors. Previous shrinkage methods tend to fail when the samples are less the order of assets, but Ledoit and Wolf (2003) present a method that can be applied in practice.

The estimated coefficients in the sample covariance matrix that are extremely high tend to contain a lot of positive error and therefore need to be pulled downwards to compensate for that. By shrinking the covariance matrix, extreme values will be more adjusted than lower values resulting in a more robust estimation, and hence a better optimization.

Shrinkage is simply a transformation. The problem remains that we still don't know the true covariance matrix, hence this must be estimated before the shrinkage can be applied. The process therefore is based on 3 steps: Find shrinkage target, find shrinkage intensity, and finally calculate the new shrunken sample covariance matrix.

We won't explain here how the model works or the construction behind it, but solely provide some easy steps for implementing the method. For further information we refer you to Ledoit and Wolf (2003).

Shrinkage target

The target proposed is a constant correlation model. In short - the average of all the sample correlations is the estimator of the common constant correlation.

First, we need to find the sample correlations and the mean sample correlations:

Eq. 1.1

Eq. 1.2

Construct the shrinkage target matrix F by using the variance from the sample covariance matrix on the diagonal, and the off-diagonal adjusted covariance:

Eq. 1.3

Eq. 1.4


Shrinkage intensity/constant

This is the weight that decided the relation between the sample covariance matrix and the shrinkage target, in order to calculate the shrinkage covariance matrix. The optimal shrinkage intensity can be approximated based on the 3 variables Pi, Rho and Gamma:

Eq. 1.5

Calculate the estimate of Rho:

Eq. 1.6

Eq. 1.7

Eq. 1.8

Calculate the estimate of Gamma:

Eq. 1.9

Where the approximation of Pi is simply the sum of all elements:

Eq. 1.10

The three approximations found allow us to find the optimal shrinkage constant. Please note that shrinkage intensity must be bounded between 0 and 1:

Eq. 1.11


Shrinkage covariance matrix

The final step is to use the shrinkage intensity in order to find the shrinkage covariance matrix with the following formula:

Eq. 1.12


Results

Ledoit and Wolf 2003 showed that shrinking the covariance matrix had the following beneficial effects on empirical data:

  • Reduces turnover
  • Increases realized information error
  • Decrease the standard deviation of returns
  • Yield higher information ratio
  • Yield higher mean excess return

To verify this we set up a simple comparison of portfolio models based on a Markowitz framework, where the first portfolio uses traditional covariance estimation and the second utilize the covariance shrinkage method. A set of portfolios with various settings was run side by side on S&P500 data for 10 years. We can verify that turnover decreased with 10% on average, and the return-variance ratio improved. A sample can be seen below:


 


Quanters Group can assist you in implementing the shrinkage method in your own portfolio as well as other estimation error minimization techniques. We offer both on-the-shelf portfolio optimization products and custom developed solutions to fit your specific requirements. Please contact us for a confidential discussion.


References

Frankfurter, G., Phillips, H., Seagle, J., 1971, Portfolio Selection: The Effect of Uncertain Means, Variances and Covariances, Journal of Financial and Quantitative Analysis, Issue 6, pp.1251-1262

Jobson, J. D., Korkie, B., 1980, Estimation for Markowitz efficient portfolios, Journal of the American Statistical Association, Issue 75, pp.544-554

Ledoit, O., Wolf, M., 2003, Improved estimation of the covariance matrix of stock returns with an application to portfolio selection, Journal of Empirical Finance, Issue 10