Main Area and Open Discussion > Living Room
Planning a major programming project - tips requested
tranglos:
Also, I would like to add what I somehow forgot to ask....What steps do YOU, as coders, take when you are planning a project?
-Josh (January 12, 2012, 10:46 PM)
--- End quote ---
Step one: stop planning, go ahead and start coding. Seriously. The moment you write the first line of code you will start getting a much better understanding of (a) what it is you are making, and (b) what difficulties to expect. And the more code you write, the better your understanding will be.
Step two: plan to throw away the first attempt. All of it. Plan to change the language, to change the data storage mechanism, change everything. This goes together with step 1, because the sooner you begin, the easier it will be to let go of what you've done and start anew with a better grasp of what the project entails. I'm not saying you will throw it away; I'm saying it happens (or should happen) most of the time.
Or, if you're not quite ready to start just yet and don't like the idea of throwing away several months' worth of work, then start coding little "testbed" projects right away. Little parts of your future app, its individual subsystems. You need to know what IO mechanism fits your app best, so just write the IO code using XML or SQLite (for example) and see how it feels. Then throw away the testbed code, because it will probably be quick-and-dirty, but by then you will know what works for you, for this specific task.
That comes from my experience, YMMV, but I have seen similar advice given by truly seasoned programmers. I imagine there are pros out there who can design everything in their heads and then just do the mechanical work of putting it in code, but it certainly doesn't work that way for me. It's exceedingly hard to see all the possible pitfalls, problems, design decisions you'll have to make until you write the actual code - and then it becomes obvious. (Maybe it's just me, I suck at chess and that's kind of similar.)
I have thrown away much more code than is running in the apps I've ever published. I was designing KeyNote 2.0 "mentally" for four years, I literally dreamt about it, before I decided I wasn't up to it (and other things intervened). But most of all it was some sort of fatigue coming from several years of living with the abstract ideal of what the new version was supposed to be, without really doing anything about it.
I have another vaporware project that I've been designing since 2001. That's right, almost 11 years now :) Someone called this "analysis-paralysis" and that's quite apt. All the projects that amounted to anything have been projects that I just sat down and wrote like there was no tomorrow.
mouser:
I have found that the "plan to throw one away; you will, anyhow." philosophy is extremely insightful but often impractical.
I think the key thing to accept is that you really will learn more in the first steps of coding an application than you can anticipate with planning ahead of time. And often the very very best way to plan and refine ideas is to start writing a throwaway first test version. And throwing away the first version is probably the very very best way to produce better cleaner code.
But often that's just not a practical option -- and furthermore, I do think there is some real value in planning out an application -- just sitting down with paper and pencil and imagining all of the operations and the workflow. If you simply sit down and code until your application is done and expect that first version to be well organized you will end up in trouble.
Perhaps another way of saying this is that planning is critical -- but you can "plan" by jumping in and writing a first version to throw away, or you can plan before you ever start coding.
However the comments about "analysis paralysis" or planning paralysis, are on point too -- I have frequently gotten held up for astonishingly long delays as I ponder the "right" way to do things. Much of the time it's just wasted delay and I would have been better off just coding a paralysis. Unfortunately, the other half of the time the delay has resulted in a rethinking of the problem which I would not have been able to do if I just jumped in and started coding.
The truth of the matter is, that much of what differentiates an experienced coder vs. someone just starting out is not knowledge of a language or implementation speed -- it's developing good judgement and skills regarding how to attack a problem. And you can only learn that through experience and practice.
tranglos:
This will be a large file containing user data. There will be external files stored in their raw formats (jpeg, pdf, etc). I was debating about an already designed database system like sqlite, but also know that I have written my own file save procedures in the past and it has worked out nicely.
I was advised AGAINST XML because of the sheer complexity of implementation. There is no desire to export this data, as my application will be the first in its industry.
-Josh (January 12, 2012, 10:28 PM)
--- End quote ---
I don't think you can underestimate interoperability and ease of access. Users will thank you for it, if they ever need to access the data outside of your app.
I wouldn't say XML is complex, as long as you can use a 3rd-party parser that does the heavy lifting for you. But it is (a) relatively slow to read (b) wasteful when the tags take up more bytes than user data (c) potentially memory intensive, depending on the parser and how you use it; (d) brittle, because a single wrong byte may prevent your app from loading any of the data (that's particularly important if your users may ever fiddle with the data files themselves).
But XML is very useful and flexible when you need or expect to (a) represent hierarchical data; (b) store data of different types, including binary; (c) modify and expand the capabilities of your app or your data files. You can easily add a new tag to store a new piece of data, which earlier versions of your app may just ignore without breaking. Doing the same in custom binary files takes more work, as you have to introduce some sort of "file version" marker and check for it all the time (version 1.1, so expect a date now; version 1.2, time is now stored as UTC; version 1.3, added a comment field here...) That last thing is much easier to do with XML.
You could also design your custom "binary" format that is structured like XML, with numeric identifiers instead of tags. That worked for me once, but after that I decided SQLite was easier and faster and more capable to begin with.
SQLite is really neat in that it doesn't need a server or any complex installation (or any db license). If you use MySQL, who's going to install it, and who's going to do tech support for MySQL issues, for example? For me, the main advantage of a database is that you can read and write small portions of your data instead of the whole datafile at once. And you get a lot of built in logic for free via SELECT queries that you would otherwise have to devise yourself. Frequent disk access is an obvious downside, although you can configure your SQLite database to stay in RAM (by specifying a sufficient cache size or even explicitly copying the db to RAM).
Another downside of SQLite for me is: interminable query strings everywhere. I do keep them all in once place actually, but unless you generate all of your SQL at runtime, you may end up with hundreds of SELECT statements that are hard to maintain. If you add a column to a table, you need to go through all of them to see which ones must be modified. I hate that part, and you can of course parametrize the queries and glue them together from pieces, but the pieces still remain as string constants, and that's bad. Still, right now I wish I'd written almost all my apps with an SQLite backend, and I'll be using it for practically every future project I can think of. It's really flexible, and by far the fastest database engine on the desktop.
tranglos:
I have found that the "plan to throw one away; you will, anyhow." philosophy is extremely insightful but often impractical.
-mouser (January 13, 2012, 09:34 AM)
--- End quote ---
I agree. And there always comes a point beyond which you will not be willing to throw all the code away, because rewriting it from scratch would just take too long. That's why I'm suggesting coding the little self-contained parts first. To try out what works and how it works. You can skip much error checking or knock yourself out with exceptions if you prefer, you don't care about the UI, you can pepper the code with verbose logging and profiling, all the things that will help you learn what you are doing, without a care in the world. I suppose it is a personal thing, but it's very specifically the method that has allowed me to go back to creating working apps after a five or so years of feeling totally not-up-to-it.
But often that's just not a practical option -- and furthermore, I do think there is some real value in planning out an application -- just sitting down with paper and pencil and imagining all of the operations and the workflow. If you simply sit down and code until your application is done and expect that first version to be well organized you will end up in trouble.
--- End quote ---
That last thing is very true. But what I'm trying to say is that planning is extremely hard - for me it is, anyway. A pencil and a napkin, sure, all the time. But in my experience I don't even know very well what to name my classes and methods before I actually type it all up see what it means and how it relates to the existing code (without necessarily compiling and running it). Probably half the time I spend coding is refactoring the things that initially came out of my head. Refactoring is typically understood as a technique for maintaining code, but for me it's a huge part of creating it in the first place. I have to see the code to really understand it. Once as a kid I got a serious knock on the back of my head, maybe that's why :)
Perhaps another way of saying this is that planning is critical -- but you can "plan" by jumping in and writing a first version to throw away, or you can plan before you ever start coding.
--- End quote ---
And at least be prepared to throw away your plans :)
40hz:
The truth of the matter is, that much of what differentiates an experienced coder vs. someone just starting out is not knowledge of a language or implementation speed -- it's developing good judgement and skills regarding how to attack a problem. And you can only learn that through experience and practice.
-mouser (January 13, 2012, 09:34 AM)
--- End quote ---
So from that I guess you're saying the real difference between an experienced and a beginner coder is...experience? ;D :P
Good to know! :Thmbsup:
Kidding...just kidding. ;)
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version