Serving the Quantitative Finance Community

 
User avatar
afoster
Topic Author
Posts: 5
Joined: July 14th, 2002, 3:00 am

C++ Database programming

August 27th, 2003, 7:19 pm

Can anyone recommend some good books/tutorials on C++ database programming. I am currently using SQL Server 2000, although I also have PostgreSQL on linux, and a Oracle 8i server available.I have had a look at the MSDN stuff on ODBC programming, but this is rather messy, and I'm not entirely convinced that ADO is the way to go; for example, can a string field of an ADO recordset map directy to an STL string?
 
User avatar
csparker
Posts: 0
Joined: October 3rd, 2001, 7:53 am

C++ Database programming

August 28th, 2003, 8:01 am

Not a book or tutorial, but a set of object libraries. Try RogueWave DBTools for example. When last I looked they were fairly generic and would work with most of the major RDBMS. Performance can be a little slow if memory serves, but that it the price of genericism. The alternative is you read your database API documentation and learn a platform specific but more highly performant dialect.
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

C++ Database programming

August 28th, 2003, 10:37 am

ODBC is not really a mass market API, however if you're doing serious database work on the Windows platform it is worth the effort.However, for most work csparker's suggestion of RogueWave is quite sound. ADO does have the charm that for a lot of bog standard stuff you can knock out code pretty quickly.
 
User avatar
psychicfriend
Posts: 0
Joined: August 4th, 2003, 3:52 pm

C++ Database programming

August 28th, 2003, 1:33 pm

There aren't really any good books. Books (at least those written about specific subjects such as ADO) are always outdated and usually just rephrase the vendor documentation (i.e. MSDN) with different examples. Actually, that's not quite true: many books are "too-up-to-date," emphasizing using the latest features, which can be a compatability problem, and moreover very often really don't add a lot of benefit). If you must read books, get an account at safaribooksonline and read them there.The MS ADO documentation is great if you are using Visual Basic. That's no good because Visual Basic sucks and when I am elected President of Califonia use of VB will be punishable by fine and imprisonment. Also, the MS docs focus on using the detailed API of the ADO objects, which in this case is the wrong thing to do.Your best bet is to plagiarize well-written working code from someone who knows what they are doing.You second best bet is to plagiarize code from me. Attached to this message please find the source code for a working C++ ADO application that reads from a database, creates a new set of database tables, and then transforms streams of data from the source tables to the target tables (this is called Extract/Transform/Load or ETL). It includes a database that the program reads from in SQL server 2000 format. I will not swear that the format of the included database is compatible with this version of the application, but if you take time to dig into the code I'll take the time to extract some real data. I also include a spreadsheet of extracted data.As you will see from the code - and the opposite of what they do in all the C++ ADO documentation - the secret to ADO is that you only use one or two API functions, and let most of the work be done by straight SQL strings in your code. I like to do ADO in C++ - I usually separate all the SQL strings into a separate .h file. I have tried using more esoteric functions of the ADO API with zero luck. And the nice thing about approaching it from a SQL string point of view is that you can test all your calls in the query analyzer before you paste them into your .h file.email me if you have questions. or post them here. whatever.enjoy!
Attachments
SampleETLSpreadSheet.zip
(50.25 KiB) Downloaded 77 times
LivETL.zip
(211.5 KiB) Downloaded 72 times
Last edited by psychicfriend on August 27th, 2003, 10:00 pm, edited 1 time in total.
 
User avatar
ImamicPH
Posts: 0
Joined: December 28th, 2005, 4:36 pm

C++ Database programming

May 30th, 2007, 3:17 am

2007 - 2003 = 4 years. That's a lifetime in computer science . . .Has anything changed? What are the best resources out there for C++ and databases now?I'm in the middle of a database-based project and need to make a decision on C++ vs. C#. Speed needs to be considered.How much of a difference will this make? Is C++ essential here? Is the C++ path feasible. I have months to put into the project and I need to eventually do it C++ anyway. How hard is C++ for this?There is another thread in the Software Forum regarding some of the project.
 
User avatar
untwigged
Posts: 2
Joined: January 14th, 2006, 3:21 pm

C++ Database programming

May 30th, 2007, 4:28 pm

QuoteOriginally posted by: ImamicPHI'm in the middle of a database-based project and need to make a decision on C++ vs. C#. Speed needs to be considered.How much of a difference will this make? Is C++ essential here? Is the C++ path feasible. I have months to put into the project and I need to eventually do it C++ anyway. How hard is C++ for this?With a typical database application the main speed issue is with the queries performed on the server - the choice of programming language makes very little difference. (There are some exceptions to this, such as using bulk insert APIs which may only be available on one platform).Having worked with C# (Ado.net) and a few C++ libraries for database access, I cannot strongly recommend C# enough. In terms of getting up and running fast, and having the widest database support through modern APIs, there's little that can beat it. It's pretty fast too, especially if you use the native sql server classes. I tend to do most of my work using the OleDb classes though, as these support different database types out of the box - still pretty efficient as ODBC isn't used.
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

C++ Database programming

May 30th, 2007, 5:47 pm

That's a good point, and it leads to another one that an SQL databases on server pretty much the worst possible choice for tick data.If you are asking "find me the most recent quote", or "find the average price over this period", they are great because you fire one packet across the network and get one back.But if ou are doing serious time series that's a large number of trips around your corporate network, you can easily lose 5 millisecond on that, maybe a lot moreThat doesn't sound much, but if you are processing every row, that can turn a sub second calculation into one that takes 15 minutes.
 
User avatar
alexv
Posts: 0
Joined: March 14th, 2005, 9:45 pm

C++ Database programming

June 11th, 2007, 4:39 pm

There is Sql module in Qt libraryhttp://doc.trolltech.com/4.2/qtsql.htmlIt is part of the free edition (Qt Open Source Edition) as well as two commercial editions - the Qt Desktop Edition (everything), and the Qt Console Edition (no GUI stuff)You will have the additional benefit of having cross-platform code.
 
User avatar
martinlukerbrown
Posts: 0
Joined: July 3rd, 2006, 7:51 pm

C++ Database programming

June 11th, 2007, 8:30 pm

Yes performance of tick data will suck on a relational engine. The main problem is that there is no implementation of a native sql type for timeseries data, this is just chatted about by relational vendors. However if you want to stick with a relational engine then X10 have a timeseries engine built on top of Oracle, or you could go the whole hog with vendors like kx or xeno.As far as C++, dbtools is good. However if memory serves its tied to specific versions of the databases client API, and you have to wait for RW to update it. I think your find the relational engine the limiting factor, so the only reason to go with C++ is if you have a huge amount of code in C++. If you need to stay in C++ land and want SQL server then you looking at oledb, which developers hate due to the number of interfaces produced, again its years since I've touched that so might of changed.
 
User avatar
bluetrin
Posts: 2
Joined: September 9th, 2005, 6:41 am

C++ Database programming

June 13th, 2007, 6:49 am

QuoteOriginally posted by: DCFCThat's a good point, and it leads to another one that an SQL databases on server pretty much the worst possible choice for tick data.If you are asking "find me the most recent quote", or "find the average price over this period", they are great because you fire one packet across the network and get one back.But if ou are doing serious time series that's a large number of trips around your corporate network, you can easily lose 5 millisecond on that, maybe a lot moreThat doesn't sound much, but if you are processing every row, that can turn a sub second calculation into one that takes 15 minutes.So what is a good choice for tick data for fast retrieval ?
 
User avatar
Stutch
Posts: 0
Joined: October 30th, 2006, 10:16 am

C++ Database programming

June 13th, 2007, 5:32 pm

Quote So what is a good choice for tick data for fast retrieval ?Not knowing your application I may be out of line but why not hold tick data in memory ?2 years worth of EUR/USD tick data runs to 650Mb.I know of one (non financial) system with 2 TB of RAM and most of the database is held in memory as it is static data.
 
User avatar
DominicConnor
Posts: 41
Joined: July 14th, 2002, 3:00 am

C++ Database programming

June 14th, 2007, 9:50 am

There's stuff like Xenomorph which integrate relatively well into desktop analytics. But as the man says, you can just bung it into memory, a few GB of RAM is pretty close to free these days.What sort of queries will you be running ?
Last edited by DominicConnor on June 13th, 2007, 10:00 pm, edited 1 time in total.