You know that particular part of your code that is essential for the project but will probably take a lot of time to get it done? Do you ever get the feeling that you’d rather work on something else (probably less important) or not code at all instead of working on that part? That beast you try so hard to avoid and use every lazy trick you know to delay its inevitable implementation?
Now, I’m probably just being lazy, but I’ve always had to deal with code like that. Write something I don’t feel like writing (and it’s worse if you’re doing it for fun and not getting paid!). A large system that will take a lot of time to get it into a stage where you get back any useful results or indication of it working. How do you start coding something like that? Most people would probably suggest divide and conquer and similar architectural techniques, but this isn’t about how you do it; it’s about how you get yourself started on doing it. What’s the very first steps you’d take?
I’ll tell a story of a case in which this happened to me.
I wanted to implement a new frametype decision algorithm for x264 that used forward dynamic programming (the Viterbi algorithm). But it was going to be complicated, messy, ugly, and so forth. And I really didn’t want to do it. I tried to pawn off the project onto Google Summer of Code, but out of some sort of terrible bad luck, the one student that we had that simply bailed on his project… was the student that chose that project.
So after two months of complaining about it and dodging it, I finally got to work on the algorithm. And here’s how I did it.
First, I talked to the other developer, who apparently already had some ideas on how to do it. We talked it over and he explained it to me until I fully understood the process from an algorithmic standpoint. This is the first step of any such project: understand the algorithm behind it so well that you can pseudocode the entire thing.
Then, I talked to another coworker of mine. We went up to a whiteboard and I sketched it out until he understood it too. By explaining it to someone else, I gained understanding myself. This is the second step: explain the algorithm to someone else so well that they can pseudocode it. This is an emulation of the programming process, since programming is a form of ‘explaining’ an algorithm to the computer.
Then, I wrote a simple Java prototype that used arbitrary fake values for the cost function and was solely being used to test the Viterbi search. I finished it, and checked it against an exhaustive search–it matched perfectly. My dynamic programming was correct. This is the third step: write the simplest possible form of the algorithm in the simplest possible environment.
Then I ported it to C, x264’s native language. It worked again. This is the fourth step: port that simple form of the algorithm to the full environment.
Then, finally, I replaced the fake cost function with the real one. After some bughunting and fixing, it worked. This is the final step: integrate the algorithm completely with the environment.
This process took barely a week, but from the perspective of me at the start of the project, it was completely daunting and I couldn’t get myself to even get started–yet by breaking it down into such a step by step process, I was able to not only get it done, but get it done much faster than I expected.
And the benefits went far beyond x264; I now understand Viterbi so thoroughly that I now can explain it to others… and those others can benefit greatly from it. For example, one of the ffmpeg developers is using an adaptation of my algorithm and code to optimally solve a somewhat different problem: optimal header placement in audio files.