Does anyone know of any good resources, web or book, for those getting involved in writing larger software systems with real complexity?My background is physics, which has given me some experience of coding medium size programs, but of the numerical type which has (I think) bounded complexity. I enjoy programming a lot, and think that there is a lot of satisfaction and challenge in making a system really well structured and well designed. As a result I have spent some hours poring over the design pattern and style manuals, and soaking up various Guru tips. While all of this has no doubt improved my coding, I still feel that maybe I can do better when approaching a large design problem. The problem is that these books are all couched in rather (compared to finance) simple systems examples. Maybe it is just my imagination, but I keep thinking that I could do a better design job with some helpful hints.I am sliding into developing bigger stuff, which I would like to be involved in, but unfortunately I am working more or less alone so I don't have any more experienced colleagues to feed off. Furthermore the few contractors we have who have more (time) experience seem to be graduates of the hack it 'til it works school.I wonder if anyone saw a book of the kind "Diary of Real Systems Developer"?

I don't have a perfect book matching your request, but one at least related book is "The Pragmatic Programmer" by Dave Thomas and Andy Hunt. I think there are plenty of useful topics covered who really helps you get a good feeling what "good" programming really is.I totally agree with you on the satisfaction and challenge of making a system beautiful and working, and even more so when the system is used for an interesting task.

maybe you shld look at computer science books on system analysis and system design, if you are developing a commercial application from scratch. understanding system analysis and design is essential.

Do not expect to learn much from books.There are so many ways to do each level of a larger system that only experience really helps.You are better stay conservative and keep you system small, buy a system that does what you want, or not do the project.

Rocket >> Do not expect to learn much from books.As far as regards IT, never a truer word was spoken. Sadly most IT books are written not to transfer knowledge but cash, and I don't buy many. Meanwhile those of high quality do not cover the whole field, maybe they can't.Rocket >> not do the projectNo such choice (joblessness). But anyway I want to dammit (childishness). I've seen enough systems to know I can do better (hubris). This is my problem; I need to absorb experience but none is to hand. I have to do it, but I want o do it well.Patrik << Thanks I've ordered "The Pragmatic Programmer".Paul << (If you're listening) This is the rub, there are many books on programming for finance, there are many books on quant finance, but nothing exposing the raw coal-face. What we need is some revealing (and uplifting) stuff on the real day to day work. I don't know if you're interested in the programming side, but you could surely handle the quant side. Enough of PWoQF, now for Paul Wilmott IS Quantitative Finance : (sub-title) Diary of a high-flying hard-working quant. Scope there for many more volumes .

Onuk, Rocket,I have to disagree with your evaluation of IT books! In my experience, the best way to find information on proven software design/coding techniques is through the following sources:1. Software programming magazines2. IT books written by experienced authors3. Good internet community sites/discussion groups such as this one4. Software development experience - your own and others' - the good and the bad.Although IT book publishing is big business, software technology evolves as such speed that a professional developer must consider all sources of information to stay up-to-date, and publications (books/magazines/electronic) are still the quickest means of delivering the information. I agree that nothing beats experience, however it takes time to acquire it and you'll get even more out of it by comparing it to other people's approach to similar problems.There are lots of IT books out there and some are definitely better than others. I've never found a single book which satisfied my search for the 'best design techniques' - I've got my list of 'favourite' authors, and try to keep track of what they publish, knowing I'll probably get good advice from their work.As far as 'hack it 'till it works' approach, the reality is that unless you are in academia or live on some other planet, there are times when you just will have to do just that - i.e. when you don't have the time to seek the the time/information you need and your deadline is yesterday! Hopefuly, with experience you will know how to do it with minimal risk, document what you've done and be aware of potential consequences.My last comment: Keep it simple - don't get too caught-up in architecting the optimal design for your project. And if you can, collaborate with others - discuss you approach with other developers and get the benefit of their experience!B.

I'd definitely recommend "The Pragmatic Programmer" - very good book, and if you stick by their advice you won't go far wrong.For other IT books, I'll agree that there's a lot of guff, but there are also some pearls. For decent reviews, look at www.accu.org - I'm a bit biased since I review for them as well, but generally they're good and fair.More practical advice:* Keep it simple: Don't make things harder than they need to be. Quite often, a really simple implementation will work just as well, and you can always revisit it later.* Keep it modular: try to write several small systems that work together instead of one single big one* Test, Test, Test, Test....!!!! Have a look at UnitTests - having a collection of unit tests that you can run after any change to check that everything still works helps a lot. * If possible, try and work in a high-level language. I love C++, but I can accomplish much more in Python in the same time.Also, if you get stuck, you can always post more specific questions.

OnukTry Analysis patterns - FowlerDesign patterns - Gamma Helm Johnson Vlissides (Gang of Four) for design insightsCode complete Steve McConnell, dealing with a lot of the issues that you will face when producing large software productsLarge Scale C++ software design - Lakos Useful if your program is in C++, helped me loads to understand how to code big projects without falling into nasty design traps that are not a problem for small programs but make bigger projects fail.I have a similar physics background and needed to become C++ and design literate very quickly the above helped along with books like 'more effective C++' (can't find author at the mo.) and Advanced C++ - Coplien which improved the quality of my code and OO design. Different people find books more helpful than others. I thought Lakos was great because it solved the problems that I was having at that time whereas my colleague thought Code Complete was great but I found it slightly too chatty. Good luck in your endeavours!!Nig

PinballWizard,Of course books are valuable to learn. I just believe they rarely help when you actually get down to doing a software project.Read books... write prototypes: yes.Read books... start a project: a waste.The real issue is that too much information that is not really digested is no better then not enough that is well understood.My technique: reread the same books over and over but try not to introduce what you have learned until you are forced to.

Rocket,Ok, I guess everyone's got their own style.I rarely/never read a computer book from cover to cover - I usually search for what I need then, and then shelve it until I'm stuck and need some ideas. I've been saved many times by old magazine articles (we're talking up to 1 - 2 yrs old!) which contained the nugget of information I was looking for.Cheers,B.

PinballWizard >> I have to disagree with your evaluation of IT books!Then you must be wrong ! Seriously, I wasn't panning all books, I have a respectable collection too and agree I got some invaluable info from them. I just think that when I look at the book store shelves there is also a lot of garbage out there. My real gripe is that although I have found many useful books on general coding tips, design techniques and so on (Design Patterns, Effective C++, etc etc) and a few on technical details, the real raw stuff is generally under-represented. For example I have seen many books on Excel & development, but never one which contained anything not in the help; for this I relied on colleagues, which is OK until you have no colleagues who know what you need to know.Nig >> Large Scale C++ software design & Code Complete Thanks, I'll have a look; I read the others already. By the way Effective C++, More EC++ and Effective STL are by Scott Meyers.

PinballWizard >> 3. Good internet community sites/discussion groups such as this one.I forgot to ask, do you know of any site such as this one focussed on programming (C++)?

see www.Codeproject.com (MS C++)There arwe some rather well known names there.

i may sound that i am soliciting for this book, by saying this, but i like the books by wrox publication. usually, about half of the book gives you the idea how on what you can do with the programming language, and how to leverage on built-in objects; while the other half summarizes in a more friendly and concise manner, what normally the documentation that you can find in sites such MSDN.but again, if you are looking for books that teach big system design and implementation, you shld be browsing those which are rather academic in nature.

I agree - the WROX books are usually pretty good. I wish they'd do something about the cover pictures though!