Hello All,Does anyone have any experience in optimizing a nonlinear objective function subject to linear constraints? I am trying to optimize a portfolio in C++ and am looking for algorithms to efficiently find an empirical solution for the Markowitz portfolio. Does anyone know of any very reliable opensource libraries that contain efficient algorithms to find the solution. I found Opt++ at http://csmr.ca.sandia.gov/opt++/OPT++2. ... index.html, but it did not support windows. Also, it needs to be compatible with the Borland C++ complier. I tried numerical recipes in C at www.nr.com, but the simplex method given was for unconstrained optimization only.If need be I am will to buy a commercial software. Any recommendations would be greatly appreciated. Does anyone have experience calling Matlab from C++? Are there any large pitfalls or steep learning curves? In general, how much work is it to set up a function to pass a covariance matrix and the constraint equations to Matlab and expect a quick and easy solution? Thanks in advance for any advice.Gabriel

Sounds interesting. I hope you find an elegant solution.

Here is the shortcut, try to use excell solver.

Hi leptoq1Excel solver is OK for a limited number of assets.I have found it struggles with a higher number of assets, say 200.I am not sure exactly where it fails. Possibly around 50 assets.What have you found?

Greg: I think there is a professional edition of solver for larger scale optimisation problems (industrial applications) Gabriel: Also check for GAMS and Palisade's tools.

Yes you are right. I was thinking more of the standard solver.Any idea of how long it would take to optimise, say 200, 300, 500, 1000 and 5,000 assets using one of the super-charged solvers?

Excel solver is not going to work;; keep the matlab approach.It is faster and more accurate. My master thesis was a comaparison of ANN vs regression approach when the objective function was a a MIMO system and nonlinear/ MAtlab is the way to go

QuoteOriginally posted by: GregWallaceYes you are right. I was thinking more of the standard solver.Any idea of how long it would take to optimise, say 200, 300, 500, 1000 and 5,000 assets using one of the super-charged solvers?I have no idea since i have never been involved so far with such projects, and its been some time now since i used to read OR journals. Anyway i guess vendors' websites can provide some relevant info not far away from reality. I also guess that its not only the number of assets(variables) but also the number of constraints that plays some role in the time function.

QuoteOriginally posted by: gabrielgrayHello All,Does anyone have any experience in optimizing a nonlinear objective function subject to linear constraints? I am trying to optimize a portfolio in C++ and am looking for algorithms to efficiently find an empirical solution for the Markowitz portfolio. Does anyone know of any very reliable opensource libraries that contain efficient algorithms to find the solution. I found Opt++ at http://csmr.ca.sandia.gov/opt++/OPT++2. ... index.html, but it did not support windows. Also, it needs to be compatible with the Borland C++ complier. I tried numerical recipes in C at www.nr.com, but the simplex method given was for unconstrained optimization only.If need be I am will to buy a commercial software. Any recommendations would be greatly appreciated. Does anyone have experience calling Matlab from C++? Are there any large pitfalls or steep learning curves? In general, how much work is it to set up a function to pass a covariance matrix and the constraint equations to Matlab and expect a quick and easy solution? Thanks in advance for any advice.GabrielGabrielI have used differential evolution and I find it a very powerful method. The web site has ports to a variety of languages and anway it should not take too long to code your own version up. The basic idea of it is pretty straightforward and it can be parallelised.Mark

check ASA codeAdaptive Simulated Annealing

I tried excell solver (probably professional version), it worked fine for 2000 variables it took around 1-3 minutesfor the problem that I was looking at. I used it only for QA of something else. Obviously, this is not a good solution. If you want to have a good nonlinear optimiser with constraints, you'll have to write your version of software which will be customised for your specific problem. In general the danger is that you'll be stuck in a local minima/maxima, so that my advice is not to try to solve the generic problem. For example in some cases nonlinear (non quadratic) optimization problem can be reduced to equivalent linear problem or quadratic programming, which has well developed algorithms.Here is another suggestion which worked for me around 3 years ago. 1. Find an impementation of Levenberg - Marq. algorithm in www.netlib.org collection of open source numerical libraries ( I don't remember exactly but I think Matlab optimiser is based on the same code). 2. Provide a pointer to a function which defines your optimization problem3. Implement constraints either as penanlty functions (or change variables ...)It easy to give advices, this project requires a lot of work, good luck!

If you want to have commercial solvers I really can recommend GAMS or Matlab (ML only together with Tomlab!) (and I do have some years of experience in optimization ).If you do not want to spend some money, look at http://plato.la.asu.edu/guide.html Both Hans Mittelmann and Peter Spellucci are real experts and they have links to the best free software on this page.And, please. do not solve Markowitz problems with more than 20 (up to 50) assets. Did you ever had a look on a) your data, b) the condition of your covariance matrix? Please be very careful about what you do.

Condition number of Covariance matrix will be huge (if the length of time series used for its calculation is smaller than number of assets in the optimization problem), but it does not mean that optimization problem should not be solved for large number of assets, does it? It means that naive calculation of covariance matrix should not be used as an input to the optimization, since your Markovits weights roughly speaking are dominated by eigenvalues of inverse covariance matrix (this is just noise). But it looks like this is a story for a different thread.

I think it partly fits in the thread: if you want to solve large CQP (convex quadratic programs, and Markowitz is such) your solver needs to be able to deal with large and not too well conditioned matrices, if the algorithm e.g. uses Hessian information. If you run some optimization with active set strategies, this problem is not that pressing as only a very small part of all constraints is used in each iteration. Nevertheless, what I wanted to point out was: large scale optimization needs different solvers than mid scale problems and the latter are much easier to solve.If I have it in my own decision if I solve small or large problems, I have to think carefully whether a two step procedure - e.g. first optimization in sector indices and second, optimization within each sector - using simple optimization routines will give me only an epsilon-worse solution compared to the original large problem.In the other part you are right: I have a very strong opinion on solving Markowitz for more than 20 - 50 assets - but this for sure is the wrong thread here.

I declare an interest as a vendor - www.solver.com - it would seem senseless to merely lurk without comment - so I beg forgiveness in advance.We develop and publish a range of Linear and non-linear Solvers for MS Excel and also as dll libraries.Our software has been used for many years to support Markovitz style portfolio optimisation. You can download and try our Excel stuff foc for 15 days off the website. If you need the dll evaluation, let me know, as that is emailed out upon request - including the complete user manual. There are a variety of technologies available reflecting the variety in types of problem encountered. We are quite excited about our new Second Order Conic Programming and MOSEK optimization engines for MS Excel, which offer interesting possibilities in the portfolio optimization sector. There is a huge amount of further information on the website.