Changing code on a production system to quick-fix a problem is seductive. Even if you know it’s evil and bad and dangerous – the day comes you ignore the facts and do it nevertheless.
For all of you that go to the dark side from time to time: How do you try to fix the drawbacks? Do you install a SVN (…) Server to track changes on the prod machines? Install a job that compares checksums of files and sends out “remember-you-changed-this”-Mails? Just place a note on the whiteboard? Sync changes back to a development server?
Added: I take it as a fact that this kind of bad practice happens. I am not interested in a perfect workflow to avoid this. Or whether it happens more often in PHP or JAVA or COBOL projects. Or in small vs. big projects. In newbie vs. veteran projects. Or if you get immediately punished by a cosmic entity if you do it. I am simply interested in creative usable tips from people that know how to handle that kind of situation.
Have a rollback plan in case the quick fix doesn’t work.
For a website, it may be as simple as copying the whole thing to a backup folder.
Often, this entails having a database script to undo changes made in database scripts.
Have a smoke test so you can tell immediately if you have broken the application.