wilmott.com

Hello,I'm wondering what are the methods to assess a quantitative forecast. I have a model which tries to predict short-term movements of the UST yields, and I can back-test it on historical data. Each day it gets a new value and tries to forecast the next day's closing yield.Is there any method, using which I could say whether the forecast is relatively good or I'd better throw it to trash?Cheers!

Not an expert at forecasting (believing as I do that it is pretty much a mugs game and that since so many people try someone is bound to be right, even multiple times in a row) but as a start you could assess whether the forecast was better than: i) the last observed value of the series, or ii) the implied forward.

MISE?

QuoteOriginally posted by: GuestHello,I'm wondering what are the methods to assess a quantitative forecast. I have a model which tries to predict short-term movements of the UST yields, and I can back-test it on historical data. Each day it gets a new value and tries to forecast the next day's closing yield.Is there any method, using which I could say whether the forecast is relatively good or I'd better throw it to trash?Cheers!There are so many ways. The common factor is that all of them compare the forecasts against a naive model.The most obvious as sum of squared residuals, which you can compare agasint the sum of squared residuals from the mean. Directional forecast (% - number of correct prediction regarding direction (positive, negative)). The naive approach here is 50%.Or, as DavidJN said, comparing against the random walk model (forecast today = price yesterday).There also some approaches with classical statistical validation ("if alpha lower than 5%" type of test). My suggestion is go to SSRN and search for papers in forecasting. Also, It seems to me that what you're really trying is a quant strategy of the timing type (if return forecast is positive, buy) . In that case the number of methods to check if there is any value in the forecast is much higher. Search here in wilmott and you'll find some old posts about it in the trading forum.

are you forecasting a distribution?if yes you can follow Diebold's method for density forecastsi am sure you can google for itkyriakos