SERVING THE QUANTITATIVE FINANCE COMMUNITY

• 1
• 2
• 3
• 4
• 5
• 17

tagoma
Topic Author
Posts: 18351
Joined: February 21st, 2010, 12:58 pm

### Python tricks

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Python tricks

You're welcome!But that algorithm is a hack because it does not use nice vector commands. (Loops are for losers!)Here's better pseudocode (assuming python has some way to extract a shorter vector from among the elements of a longer vector)1. Create a vector of {+1,-1} corresponding to the sign of the price change2. Create the cumulative sum of the signs3. Extract the cumulative sum values that occur where the signs change sign4. The first differences of this smaller vector was the signed run lengthsNote 1: step 3 is a bit tricky because you need to get the index alignment of the logical test just right (including the end-of-vector case). I think you'll want to extract the CumSumOfSigns[j] where PriceChangeSign[j] != PriceChangeSign[j+1]. That implies comparing the vector PriceChangeSign to a copy of PriceChangeSign shifted by 1 element (with a "0" filling the off-the-end element of the shifted vector to guarantee that an inequality.)Note 2: Depending on how you write this, the first, last, or both ends of the first-difference of the vector (step 4) will be the lengths of the partial runs at the ends of the window that you'll want to ignore in your statistical analysis.

tagoma
Topic Author
Posts: 18351
Joined: February 21st, 2010, 12:58 pm

### Python tricks

Say, I have quaterly GDP growth data, in a dictionary, pandas dataframe, or whatever tuple.What is an elegant way to get monthly interpolated data from that? (cubic spline)

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Python tricks

QuoteOriginally posted by: tagomaSay, I have quaterly GDP growth data, in a dictionary, pandas dataframe, or whatever tuple.What is an elegant way to get monthly interpolated data from that? (cubic spline)What are your accuracy/smoothness/arbitrage requirements?
Last edited by Cuchulainn on March 28th, 2016, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

tagoma
Topic Author
Posts: 18351
Joined: February 21st, 2010, 12:58 pm

### Python tricks

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: tagomaSay, I have quaterly GDP growth data, in a dictionary, pandas dataframe, or whatever tuple.What is an elegant way to get monthly interpolated data from that? (cubic spline)What are your accuracy/smoothness/arbitrage requirements?Hello Cuch. My question bears more on the dataframe creation side than on the relevance of a given computation for this Quarterly to Monthly transformation. I mean pandas probably has cool factories that can help to do that. Anyway, I'll be happy to hear advices on the 'right' computation to use. (i'm sorry maybe i just didn't get your question)

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Python tricks

QuoteOriginally posted by: tagomaQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: tagomaSay, I have quaterly GDP growth data, in a dictionary, pandas dataframe, or whatever tuple.What is an elegant way to get monthly interpolated data from that? (cubic spline)What are your accuracy/smoothness/arbitrage requirements?Hello Cuch. My question bears more on the dataframe creation side than on the relevance of a given computation for this Quarterly to Monthly transformation. I mean pandas probably has cool factories that can help to do that. Anyway, I'll be happy to hear advices on the 'right' computation to use. (i'm sorry maybe i just didn't get your question)Ah, maybe I was thinking about interpolation schemes Does Python provide them ready made?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

tagoma
Topic Author
Posts: 18351
Joined: February 21st, 2010, 12:58 pm

### Python tricks

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: tagomaQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: tagomaSay, I have quaterly GDP growth data, in a dictionary, pandas dataframe, or whatever tuple.What is an elegant way to get monthly interpolated data from that? (cubic spline)What are your accuracy/smoothness/arbitrage requirements?Hello Cuch. My question bears more on the dataframe creation side than on the relevance of a given computation for this Quarterly to Monthly transformation. I mean pandas probably has cool factories that can help to do that. Anyway, I'll be happy to hear advices on the 'right' computation to use. (i'm sorry maybe i just didn't get your question)Ah, maybe I was thinking about interpolation schemes Does Python provide them ready made?Yes, scipy provides several interpolation tools.

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Python tricks

Is overshoot an issue?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Python tricks

QuoteOriginally posted by: CuchulainnIs overshoot an issue?Indeed! Worse than that, this interpolation gets the wrong answer -- the sum of the 3 interpolated months (i.e., the integral of the interpolating curve) should equal the quarterly values. (Note even linear interpolation gets this wrong)One could convert the growth rates into levels and then interpolate.

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Python tricks

QuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnIs overshoot an issue?Indeed! Worse than that, this interpolation gets the wrong answer -- the sum of the 3 interpolated months (i.e., the integral of the interpolating curve) should equal the quarterly values. (Note even linear interpolation gets this wrong)One could convert the growth rates into levels and then interpolate.I don't see how integrals are involved here. Is some kind of preprocessing needed? Missing something?
Last edited by Cuchulainn on March 29th, 2016, 10:00 pm, edited 1 time in total.
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Python tricks

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnIs overshoot an issue?Indeed! Worse than that, this interpolation gets the wrong answer -- the sum of the 3 interpolated months (i.e., the integral of the interpolating curve) should equal the quarterly values. (Note even linear interpolation gets this wrong)One could convert the growth rates into levels and then interpolate.I don't see how integrals are involved here. Is some kind of preprocessing needed? Missing something?The data is quarterly growth data, right? If depends on whether it's an annualized rate, quarterly growth, or year-on-year. If it's an annualized rate, then you are right. If it's quarterly growth, then the monthly growth must combine to be the quarterly figure. And if it's YoY, then the constraint is even messier.

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Python tricks

QuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnIs overshoot an issue?Indeed! Worse than that, this interpolation gets the wrong answer -- the sum of the 3 interpolated months (i.e., the integral of the interpolating curve) should equal the quarterly values. (Note even linear interpolation gets this wrong)One could convert the growth rates into levels and then interpolate.I don't see how integrals are involved here. Is some kind of preprocessing needed? Missing something?The data is quarterly growth data, right? If depends on whether it's an annualized rate, quarterly growth, or year-on-year. If it's an annualized rate, then you are right. If it's quarterly growth, then the monthly growth must combine to be the quarterly figure. And if it's YoY, then the constraint is even messier.So, what is the root problem? GIGO?
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Posts: 23951
Joined: September 20th, 2002, 8:30 pm

### Python tricks

QuoteOriginally posted by: CuchulainnQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnQuoteOriginally posted by: Traden4AlphaQuoteOriginally posted by: CuchulainnIs overshoot an issue?Indeed! Worse than that, this interpolation gets the wrong answer -- the sum of the 3 interpolated months (i.e., the integral of the interpolating curve) should equal the quarterly values. (Note even linear interpolation gets this wrong)One could convert the growth rates into levels and then interpolate.I don't see how integrals are involved here. Is some kind of preprocessing needed? Missing something?The data is quarterly growth data, right? If depends on whether it's an annualized rate, quarterly growth, or year-on-year. If it's an annualized rate, then you are right. If it's quarterly growth, then the monthly growth must combine to be the quarterly figure. And if it's YoY, then the constraint is even messier.So, what is the root problem? GIGO?It's not GIGO. It's an extension of what you said in your first reply. The choice of a interpolating function depends on the structural properties of the system. For example, should first derivatives, second derivatives, .... be continuous across the data points? In the case of some kinds of growth data, the constraint is on the first integral of the data.

tagoma
Topic Author
Posts: 18351
Joined: February 21st, 2010, 12:58 pm

### Re: Python tricks

I just came across Dash the new interactive Python framework developed Plotly. It allows to make cool apps. (Notice I am not connected in any way with them)

Cuchulainn
Posts: 61108
Joined: July 16th, 2004, 7:38 am
Location: Amsterdam
Contact:

### Re: Python tricks

Here is a nice trick I just did
# This program prints Hello, world!

print('Hello, world!')
http://www.datasimfinancial.com
http://www.datasim.nl

Every Time We Teach a Child Something, We Keep Him from Inventing It Himself
Jean Piaget

Wilmott.com has been "Serving the Quantitative Finance Community" since 2001. Continued...

 JOBS BOARD

Looking for a quant job, risk, algo trading,...? Browse jobs here...

GZIP: On