Guys, this is probably a simple question for you. I need to come up with a function that would estimate the quality of strategy based on the total number T of its signals and the number of its correct signals, C<=T.I think a straightforward approach f(C,T) = C/T is not the best approach since the value of this function will be the same for the following two strategies:1) T=3, C=22) T=30, C=20Obviously, I would prefer the second strategy as it is sort of more statistically significant due to a higher number of signals, and I want it to be taken into account in a generic way.Any ideas? Thanks!

Well, there are various ways to fool yourself with your measure, but putting that aside, what you are seeking is called the confidence intervalregards,

Your rationale for your preference for strategy two says that you are risk averse (strategy #1 is just as likely to be better than strategy #2 as it is to be worse than #2). The question is: how risk averse are you? This will determine the confidence interval that you use.There are three other issues to consider:First, you might want to analyze your strategies' patterns of correct signals. If #2 produces 20 correct signals in a row and then 10 incorrect signals in a row, it's probably a bad strategy. See http://en.wikipedia.org/wiki/Wald-Wolfowitz_runs_test for a lead on one such test.Second, analyzing the % of correct entries only makes sense if all strategies produce identically distributed profits when they are correct and identically distributed loses when they are incorrect. If strategy #1 generated two gains of 10% each and one loss of 1%, then one might prefer that to strategy #2 if strategy #2 generated twenty gains of 1% each and ten losses of 3% each. Thus, you might be better off analyzing the mean and standard deviation of alternative strategies (and computing confidence intervals of those values).Third, you might think about how many entries you want per unit of time relative to your holding time and allocation of capital across other trading strategies. Leaving aside the issue of statistical confidence in the results of strategy #1 vs. #2, you might prefer a trading strategy that generates fewer entry signals than #2 does. Or, you may decide that neither #1 or #2 utilize you capital to the full extent. You might consider trading both strategies at the same time (with some allowances for differences in risk between the two strategies and the correlation between the two strategies).

Guys, thank you for your posts! I decided my approach to strategy optimization to be step-wise. First I come up with some indicators that generate entry signals. Each such an indicator has its own parameters that I want to optimize using the objective function that I am asking you about. Once I have the signals I will apply capital allocation and money management scheme on top of it and optimize considering other measures like sharpe, stirling, retracement ratios, etc. With such a step-wise approach I hope to to achieve more stability and less curve-fitting as I will have more stages of optimization but fewer parameters in each stage as opposed to optimizing over all parameters in one go.QuoteSecond, analyzing the % of correct entries only makes sense if all strategies produce identically distributed profits when they are correct and identically distributed loses when they are incorrect. If strategy #1 generated two gains of 10% each and one loss of 1%, then one might prefer that to strategy #2 if strategy #2 generated twenty gains of 1% each and ten losses of 3% each. Thus, you might be better off analyzing the mean and standard deviation of alternative strategies (and computing confidence intervals of those values).Yes, I am aware of that. By applying a fixed grid (like stop-loss/profit target) after the entry to the market or something like a trailing stop I can ensure that potential loss resulting from any entry is limited. This gives me uncertainty on the up-side only and I will try to capture it with proper money-management scheme afterwards.QuoteThird, you might think about how many entries you want per unit of time relative to your holding time and allocation of capital across other trading strategies. Leaving aside the issue of statistical confidence in the results of strategy #1 vs. #2, you might prefer a trading strategy that generates fewer entry signals than #2 does. Or, you may decide that neither #1 or #2 utilize you capital to the full extent. You might consider trading both strategies at the same time (with some allowances for differences in risk between the two strategies and the correlation between the two strategies). Agree. I was thinking of introducing a third argument to the function - the expected frequency of trading and then penalizing the score of the strategy based on the deviation of the actual signal frequency from the desired one.I will definetely look at Wald-Wolfowitz test.Many thanks!