Tod1d's Thought Process

Nearly 4 weeks ago I started work on "phase 2" of an in-house application. Over the course of three weeks many features were added, several enhancement requests were completed, and many a bug was stepped on, in some cases, repeatedly.

My hope is that by writing a postmortem I will benefit from summing up the good and the bad, by making use of the "what went right" in the future, and leaving "what went wrong" in the past.

What Went Right

Iterative releases

Early in the project, it was decided that at the end of each week (Friday), I should push the application to the staging server and e-mail the QA guy with a list of "completed" (more on this shortly) features. Included in the e-mail was a description of each feature, and instructions on how to test the feature, with guidelines as to the expected results.

I felt it was very important that the QA guy understood to test nothing but these new features. Weekly releases had the benefit of not overwhelming the QA guy with a list a mile long compiled over the span of a month (or two). This directly benefited me as a programmer.

Due to the short test period, I usually had a complete list of bugs within a day, maybe two. When I looked at a bug from a feature I added a few days earlier, which was still fresh in my mind, it often took very little time to determine a solution. Compare this to looking at a bug for a feature you added 3 weeks ago. You don't remember adding the feature, much less why on earth you decided to use a DataTable rather than a custom collection of business objects.

I did not create a link / reference, in the application, to a feature that was not listed as "completed". There is no sense in confusing the QA guy (or anyone else) by displaying a link / button that doesn't do anything. It is a bug waiting to be written up.

One of the most helpful things that I decided to do was to not put a feature on the list of completed items unless I (the programmer) was convinced that I would be willing to push that feature to production. This meant that the UI was complete, the feature had been unit tested by myself and at least one other person, and that I had "QA guy" proofed it.

If you are new to an organization with a QA guy, then it may take you some time before you will know what I mean. However, if you have been working with a QA guy for awhile (hopefully the same one), you will get to know their wily ways. You will learn what they deem important, and what crazy stuff they will do in an attempt to crash your application. Although most programmers hate to admit it, there is a chance that a user will enter a tilde (~) in the textbox that is requesting a date value, or some such nonsense.

Once you begin to learn what your QA guy looks for, by all means, write code to thwart his attempts to ruin your day. This saves an incredible amount of time for both you and him.

Another benefit of iterative releases is that it let everyone know exactly where we stood with a releasable application. If we had to release today, we knew which pieces were ready to go.

Track database changes

I kept a list of database object changes in a command file that could be easily executed to apply the changes to production. I also tried to make sure to document the reason for any changes or additions to the database objects.

Labeled release in VSS

Once the application was released, I Labeled the project in Visual Source Safe. Labeling is simple and allows you to return to a specific point in the application's lifecycle. This is useful when you need to quickly restore the application from a previously known working version.

What Went Wrong

Didn't test "unchanged" sites

The morning following the push revealed an error which affected two "unchanged" sites. These "unchanged" sites share the same database as the sites that had been pushed the night before.

After the database changes took effect in production, a feature which had been working fine was now broken. During the push, a User Defined Function's (UDF) signature had been changed to require two arguments, rather than one.

Before the push to production I was sure that I had updated all references to the UDF to pass in two arguments. It seems, however, that I still managed to miss a single reference buried deep in a stored procedure, of an loosely related database.

Which brings me to my final point.

Broke the contract

As I said in the previous paragraph, I ran into a problem when I changed the signature of an existing UDF. Several applications outside of the two I was working on used this UDF. I thought I had checked thoroughly for all references to this UDF. I missed two references, and as a result, application modules broke. This is called breaking the contract.

Think of a broken contract as a divorce where the husband and wife nullify there relationship without telling the kids. The kids wake up the morning after the divorce and get into an argument over whether or not Spider-Man could destroy Batman in a fight. They soon realize that their father, who is an avid comic book reader, would have to resolve this. They run to their parents' room and knock on the door. The door opens, but instead of finding both parents, they find their mother alone. The mother explains that dad is no longer going to be living here, and the kids world begins spinning out of control.

If you didn't catch it, the marriage (contract) is the function. Once the parents divorced (broken contract) the function's signature changed. The kids (object calling the function) expected that the contract was still in place. When they find that it isn't, their world collapses.