Since test automation scripts would run in most of the cases in scheduled mode unsupervised without any manual intervention, it becomes imperative to have solid tracing and logging mechanisms built into the automation framework so that if anything goes wrong, it is still possible to recreate the error and make meaningful logical inferences about the source of the error.

The Long Story

All remains well when nothing goes wrong. But that unfortunately is not the case in the real world. In the real world, disks get full, memory becomes corrupt, file-handles become invalid, power goes off – in short – so many things which “should not happen, indeed happen. This reality of our world is best expressed by Murphy’s Law – “If anything can go wrong, it will.”

And when things go wrong, we want to set them right. Further, being human, we would want to know the circumstances which led to the error and take precautionary measures for the future.

Such “Holmes”-like an investigation is never easy. More often than not exact sequence of events leading to the error or unknown. In other cases, the same conditions do not give the same error!

So what could aid the poor test-developer in debugging such errors when they occur?

Early investment in tracing and logging mechanisms

While the choice of exact mechanism for logging and tracing will depend on multiple factors like platform of development, underlying operating system, language/compiler support, system constraints amongst many other, there are few guidelines which could help in making the right choices.

First and foremost, if tracing and logging effort is included in architecture and design, this in itself is a big right step. This would ensure that implementation is natively available when required.
On platforms like .Net and Java, tools like log4net , log4j, slf4j, Enterprise Library could be of immense use. These are well crafted feature rich libraries which make the task of logging a breeze. Before trying to “re-invent the wheel” – it will be a good idea to explore these libraries and ascertain requirements against their feature sets.
As we have discussed before keeping the logging implementation simple is a good idea.
Keeping in mind the probability of Murphy’s Law, conditions which should never occur, should always be logged.
It will be worth-while to invest in a screenshot utility so that errors are captured at UI level. Experience says that these screenshots become immensely useful evidence in debugging process later.
All popular logging frameworks have notion of log-levels and while writing test-logic it is imperative to use the log levels judiciously. Too less of log will be lost in woods, too much of log will be noise. Log levels help by providing a throttling mechanism so that at runtime we can tweak what messages we really want to be logged and what we want to be filtered depending on our current situation.

Not all good things in a developer’s life are free. Investing in tracing and logging mechanism is in reality an architectural decision which should get translated in non-intrusive design elements, further making its way into actual implementation. If tracing and logging is not given its due importance, and is meted with second-grade treatment by adding as an after-thought to the implementation, the results will fall short of desired and will not be adequate enough to be a strategic aid in debugging.

Continued…