QuoteA few thoughts on filters:1. The simplest filter would be a moving average filter with one zero. (Not a pole!). This is a non-recursive filter.A simple moving average (SMA) is a good example.2. The simplest filter with one pole would be an IIR filter of order 1. An exponential moving average (EMA) is a good example in this category.3. The frequency cut-off of a filter is only one characteristic. The other equally important characteristics are delay and phase response. Delay is perhaps more important in the financial world. You dont want to act on information thats too stale. One reason for using an EMA rather than an SMA is that the EMA gives a better smoothing than the SMA for a given delay. Alternately EMA gives a lesser delay than SMA for a given smoothing.Phase response dictates how dfferent frequencies are delayed. Ideally the phase reponse should be a linear function of the frequency. This would mean that all frequencies are delayed by the same amount and the signal is not distorted. In communication and signal processing literature this effect is often referred to as dipersion.4. Finally be aware of the differences in notation used by the EE and Statistical communities when treating recursive filters. The EE community notations would lead to the requirement that all poles be within the unit circle for the filter to be stable. The Stat literature often requires all poles to be outside the unit circle. Thi arises due to the way poles and zeroes are defined.My 2 cents.Hedge Quant. hedgeQuantYou're obviously not an EE, since nothing you said is correct. Johnny boy has enough trouble as it is.N

QuoteOriginally posted by: N<blockquote>Quote<hr>A few thoughts on filters:1. The simplest filter would be a moving average filter with one zero. (Not a pole!). This is a non-recursive filter.A simple moving average (SMA) is a good example.2. The simplest filter with one pole would be an IIR filter of order 1. An exponential moving average (EMA) is a good example in this category.3. The frequency cut-off of a filter is only one characteristic. The other equally important characteristics are delay and phase response. Delay is perhaps more important in the financial world. You dont want to act on information thats too stale. One reason for using an EMA rather than an SMA is that the EMA gives a better smoothing than the SMA for a given delay. Alternately EMA gives a lesser delay than SMA for a given smoothing.Phase response dictates how dfferent frequencies are delayed. Ideally the phase reponse should be a linear function of the frequency. This would mean that all frequencies are delayed by the same amount and the signal is not distorted. In communication and signal processing literature this effect is often referred to as dipersion.4. Finally be aware of the differences in notation used by the EE and Statistical communities when treating recursive filters. The EE community notations would lead to the requirement that all poles be within the unit circle for the filter to be stable. The Stat literature often requires all poles to be outside the unit circle. Thi arises due to the way poles and zeroes are defined.My 2 cents.Hedge Quant. <hr></blockquote>hedgeQuantYou're obviously not an EE, since nothing you said is correct. Johnny boy has enough trouble as it is.NWhat exactly is not correct ?hedge Q

What exactly is not correct ?1, 2, 3 & 4.For example, a filter with one zero is a high-pass filter (like a treble control) and is not a moving average filter.Where did you dream-up SMA and EMA??

With all respect to you N, it does not seem that Digital Signal Processing is your strong point.Consider the filter y(k)=x(k)+0.5*x(k-1)where x(k) is the input and y(k) is the output. This is an MA(1) filter with a single zero. What do you consider this to be ? High pass or low pass ?I have attached a zip file containing the frequency response of the filter (in both eps and pdf formats).Does that look like the frequency response of a low-pass or hi-pass filter ?Now consider the filter:y(k)=(1/N)[x(k)+x(k-1)+.....x(k-N+1)]This has N zeroes and is a Moving Average filter. Do you propose to argue that moving average is a high pass filter ? This is the so called Simple Moving Average (SMA). If you dont know the term look up a good book on Technical Analysis.Now consider the filtery(k)=a*x(k)+(1-a)*y(k)This is the Exponential Moving Average (EMA) filter. This is a recursive filter and has a non-trivial pole and a trivial zero. If somebody else is interested I will post the delay for the SMA and EMA filters. I will also be happy to post the matlab code used for generating the plot.Hedge Quant.

QuoteConsider the filter y(k)=x(k)+0.5*x(k-1)where x(k) is the input and y(k) is the output. This is an MA(1) filter with a single zero. What do you consider this to be ? High pass or low pass ?I have attached a zip file containing the frequency response of the filter (in both eps and pdf formats).Does that look like the frequency response of a low-pass or hi-pass filter ?Now consider the filter:y(k)=(1/N)[x(k)+x(k-1)+.....x(k-N+1)]This has N zeroes and is a Moving Average filter. Do you propose to argue that moving average is a high pass filter ? This is the so called Simple Moving Average (SMA). If you dont know the term look up a good book on Technical Analysis.Now consider the filtery(k)=a*x(k)+(1-a)*y(k)This is the Exponential Moving Average (EMA) filter. This is a recursive filter and has a non-trivial pole and a trivial zero. If somebody else is interested I will post the delay for the SMA and EMA filters. I will also be happy to post the matlab code used for generating the plot.Hedge Quant. lpfilt.zip (17 KB) MA(1) is both a high pass filter and low pass filter, which is a notch filter. If the diagram went further infrequency you'd also see 0 db at high frequencies.You need to correct the transfer function toy(k)=0.5*x(k)+0.5*x(k-1) for MA(1)The MA(n) is also a notch filter with greater slopes (db/octave), ie. both a low pass and high pass andagain rejects the sampling frequency.With MA(n) filters, the sampling frequency must not be near frequencies of interest in the source.The RC filter or one pole low pass filter is an AR filter not an MA (my error in a previous post).Is this better HQ.Is there such a thing as a good book on Technical Analysis??N

I guess we are arguing about a notation, N. Usually MA means any filter of the form:y(k) = b(0)x(k)+b(1)x(k-1)+........ b(n)x(k-n)Certainly the filter you mentioned is also a Moving Average. In Time Series analysis MA means a general filter of the form mentioned above. Hence the filter y(k)=x(k)+0.5x(k-1) is certainly a MA filter. So is y(k)=0.5(x(k)+x(k-1)).I dealt with the filter in the discrete domain. There the frequency is always between -Pi and Pi. I normalized the frequency between -1 and 1 and plotted the response in the domain [0 1]. The response will be symmetrical in the domain [-1 0]. Outside the interval [-1 1] the frequency response will repeat itself. Understand that in the digital domain there is no sense in plotting the Freq. Response beyond [-1 1]. So to answer your question, the highest frequency is 1 (or -1). After 0 you can see that the filter response steadily falls. Hence the MA is not a notch filter as you had mentioned. In order to get a notch you need an ARMA filter. Put a zero at the notch and place a couple of poles to get a spectral high near the zeroes. This will ensure that the notch is "sharp".One more comment on the sampling frequency. A discrete filter does not "reject" the sampling frequency. If there is a component exactly at half the sampling frequency, it will appear as a constant in the sampled domain. This is an effect of sampling. The filter has nothing to do with this.If you need to go deeper into this look at a good book on DSP like Proakis and Manolakis or Oppenheim and Schaeffer.At this point I should say that all my earlier posts pertain to digital filters alone. In all the equations the variable k is a discrete variable. Hedge Q.

QuoteOriginally posted by: hedgeQuantI guess we are arguing about a notation, N. Usually MA means any filter of the form:y(k) = b(0)x(k)+b(1)x(k-1)+........ b(n)x(k-n)Certainly the filter you mentioned is also a Moving Average. In Time Series analysis MA means a general filter of the form mentioned above. Hence the filter y(k)=x(k)+0.5x(k-1) is certainly a MA filter. So is y(k)=0.5(x(k)+x(k-1)).I dealt with the filter in the discrete domain. There the frequency is always between -Pi and Pi. I normalized the frequency between -1 and 1 and plotted the response in the domain [0 1]. The response will be symmetrical in the domain [-1 0]. Outside the interval [-1 1] the frequency response will repeat itself. Understand that in the digital domain there is no sense in plotting the Freq. Response beyond [-1 1]. So to answer your question, the highest frequency is 1 (or -1). After 0 you can see that the filter response steadily falls. Hence the MA is not a notch filter as you had mentioned. In order to get a notch you need an ARMA filter. Put a zero at the notch and place a couple of poles to get a spectral high near the zeroes. This will ensure that the notch is "sharp".One more comment on the sampling frequency. A discrete filter does not "reject" the sampling frequency. If there is a component exactly at half the sampling frequency, it will appear as a constant in the sampled domain. This is an effect of sampling. The filter has nothing to do with this.If you need to go deeper into this look at a good book on DSP like Proakis and Manolakis or Oppenheim and Schaeffer.At this point I should say that all my earlier posts pertain to digital filters alone. In all the equations the variable k is a discrete variable. Hedge Q.HedgeQuant, you won't fix your problem by appealing to mathematical logic. Simply flip the switch on the blender, and you will be a happy camper.

"The RC filter or one pole low pass filter is an AR filter not an MA (my error in a previous post)."Yes, I thought you had included this as a "deliberate mistake" to see if I was paying attention. Obviously a MA just eliminates signals with the same wavelength as (or multiples of) the MA period."With MA(n) filters, the sampling frequency must not be near frequencies of interest in the source."Beats, right?Progress so far: given AR and MA we can eliminate single frequencies or all frequencies above or below a certain level. Pretty much anything I want to do in filtering using linear models can be written as a combination of these. So the next step was to re-write the ARMA stuff in state space terms to find the optimal linear filter. OK. So far so good. Next is to understand non-linear. I've got this pile of papers to read about UPF and so on. Any reading recommendations in this area?

Thanks for the advice Nonius.I am sorry that I am still missing one point in this thread. I thought that we were talking about filtering in the digital domain. In that case, where is the issue about the sampling frequency ? And how do you get the financial data in the analog (i.e, continuos time) domain ?Now regarding the use of ARMA filters, they can be designed to eliminate a single frequency (notch filters), a band of frequencies (low pass, high pass and band pass filters) or eliminate a frequncy and all its harmonics (comb filters). There are filter design algorithms which will allow you to specify the characteristics of the filter and design the optimal filter.As I pointed before, the magnitude of the frequency response is only part of the story. Before you use a filter for any procesing, be sure its phase response and delay are suitable for your requirements.hedge Q.

"As I pointed before, the magnitude of the frequency response is only part of the story. Before you use a filter for any procesing, be sure its phase response and delay are suitable for your requirements."Thanks HQ, that's good advice.

HQ,Good discussion. Now that Daxy's got my head spinning in this blender, it's harder to think straight.N

I think the distinction between digital and analog filtering is pretty artificial... at least with RC filters you can do the calculus with your fingers (i.e. don't need montecarlo or anything like that) and see the qualitative performance. Apart from sampling issues which are of second order I showed that the RC filter is lowpass or highpass depending on configuration, and the reason was that you are integrating or differentiating the input signal. What we need to get to is Johnny's 'realistic behavior' - i.e. non-normally distributed results based on simple model tricks. You know that if you don't try hard enough, the powerful magnet of central limit theorem behavior is going to suck you in and you won't get anything new. B. Mandelbrot has earned a lot of disrespect for what he's done in recent years on market asset modelling, but the one time I heard him lecture I think that there were some elements in common with this discussion. Mainly, he wanted to come up with a sufficiently simple construction which gave realistic signal behavior, since anybody can see with their own eyes that brownian diffusions aren't really that good. For hedging and other purposes it may not matter all that much, but if you're sitting there watching your nest egg jump around it is somewhat disconcerting to experience as many multi-sigma events as you do.

Regarding the dichotomy between digital (discrete-time) and analog (continuous-time) domains:The first divided difference operator, or delta operator, defined bydelta x(kT) = [ x([k+1]T)-x(kT) ] / Tallows one to easily bridge the domains. Specifically, for linear systemsdelta x = A x + B u,then, as T -> 0 we obtain a continuous-time system. This means that the two domains resemble each other to order T. But, if you use the traditional shift operator (i.e., x[k+1] = F x[k] + H u[k]) all such resemblance is lost (preciselly, it is "encoded" in the CT to DT map).Moreover filtering is much more numerically robust if you formulate things this way.See Amazon for book by Feuer and Goodwin.

"What we need to get to is Johnny's 'realistic behavior' - i.e. non-normally distributed results based on simple model tricks. You know that if you don't try hard enough, the powerful magnet of central limit theorem behavior is going to suck you in and you won't get anything new. "mmm ... my waves system generates stochastic behaviour from an entirely deterministic system so the CLT vortex holds no fears ...