Unit Testing: Beating technical debt

By Olenka Van Schendel 

On modern, distributed systems, unit testing is arguably the most effective element of your testing strategy.  A battery of well-designed unit tests “forces” the quality of your application as it is being developed – and ideally even before.  Unit test assets are then re-used continuously right across the lifetime of the application to catch any regressions in the system.

Yet how can unit testing benefit legacy applications on IBM i – and especially those that were not designed with “units” in mind, and where monolithic code has been the norm?

In this article we will examine the importance of unit testing in general and assess how relevant the technique is on IBM i.

1. Quality from Quantity  

Unit testing is often disregarded in favour of more “business-facing” techniques like functional or end-to-end testing.  Unit testing takes up valuable developer time and especially so on a legacy code base like IBM i not overly structured in “units”.  So how can a “white box” unit test, with no knowledge of the actual functionality of the application, safeguard the quality and accuracy of your system, let alone drive down your develoment costs as well?

The key is in the quantity and the coverage of the tests.

Unit testing involves creating a multitude of small, very simple test cases.  Each test isolates a particular section of code (such as procedure, subroutine, conditional block), defining success or failure in terms of the expected output parameter values for a given input.  The test run is then stored as the “base line”, and subsequent tests are evaluated against this result.  This makes unit tests very effective for detecting regressions in the system.  They can be reused, grouped and executed as part of a suite at any time, such as after every system build, after an environment change, or before a particular component is transferred to QA.  Unit tests created early on in the development phase can be used later to “smoke test” an application, to catch basic errors before any further time is wasted on more sophisticated testing.

2. Classification of defects

Indeed, typical defect classifications reveal that software defects can be attributed to a wide variety of causes, including:

• Errors in the specification, design and implementation of the application
• Errors in use of the application
• Environment issues
• Intentional damage
• Potential consequences of earlier errors

Recent surveys have shown that as many as 55% of all defects arise due to mistakes present in the specification.

It has also been observed that about 40% of a tester’s time is consumed because of the environment issues and this has a great impact on quality and productivity.

The many “peripheral” causes of defects means that overall testing costs can be significantly reduced by reusing unit test cases in “smoke testing”, before large scale and expensive system tests are undertaken.

3. Shift Left – fail early 

Unit testing is just one of a number of “developer-side” techniques for detecting defects as early as possible in the development lifecycle. Along with static code analysis, peer code reviews, code coverage analysis and other code-level practices, unit testing catches errors at the earliest possible phase, when they cost the least.

The investment in developer time to create the test in the first instance can be offset by automating the creation process. The key is to focus on typical use cases that affect the behaviour of the system. Combining both “happy path” and “edge cases” makes unit tests even more effective.

Used continuously in a CI/CD cycle, unit testing pinpoints precisely in which lines of code the defect lies. Bugs are fixed before they ever leave the hands of the developer!

4. Clean, atomic code

More than 90% of software development cost is spent on maintenance of existing systems.  Creating unit tests as you develop code actually improves the code design and makes the code easier to understand by development teams in the future.  As well as being more reliable, unit tested code is simpler, more modular, and hence more reusable.  This reduces technical debt and lowers development costs in the long term.

In its extreme form, Test Driven Development (TDM) brings extra clarity even to the definition of requirements by creating tests before the code itself.

5. What about the IBM i?

Modern IBM i applications designed with an ILE architecture already separate the business rules, data persistence and user interface into distinct modules or functions making unit test automation much easier.  The entry and exit points of a module are clear so defining a “succeed/fail” status is relatively easy.

However many applications on IBM i still contain sections of source code that were developed as long as 40 years ago and are “too important to touch”.  This creates a very precarious situation where the risk and cost of refactoring monolithic or spaghetti code into units or modules is too high to take action.  Yes the price of doing nothing is to be outpaced by more agile competition.  Luckily, unit testing creates that safety net you need as you modularize your legacy code. Generating and re-running unit tests over “backend” applications as you refactor them makes sure that previous deployments still work when combined with new functionality.

This kind of “Test Driven Maintenance” technique runs unit tests on demand and in batch as a kind of early warning system to prevent regressions emerging as you “untangle” your legacy code.

To be fully adopted by developers, any unit test functionality on IBM i must be integrated within the RDi development environment, and also with standard tools such as Jenkins and JUnit to encourage the sharing of tools between multi-technology teams.  The best unit testing solutions on IBM i can automate both the creation and execution of test cases, using parsing technology to search for parameters and their data types, rapidly identifying all the inputs and outputs of the program under test.  Cross-referencing and dependency knowledge also help manage test cases and re-use cases between versions.

6. The importance of test automation on IBM i 

There are still some extreme cases where IBM i applications cannot be easily refactored and the cost/time of restructuring the code is prohibitive.

Here functional test automation is the far more practical option.  Test cases are created automatically as users execute application functions from the standard application UI.  Unlike unit testing where the underlying data environment is dynamic, functional test data is static and restored prior to each test run to detect regressions by simple comparison of data, spool output and UI.  There is no notion of success or failure, only different.

Of course an optimal testing strategy combines both unit test and functional test automation to minimize the Mean Time to Repair (MTTR) of defects overall.  The re-use and sharing of test assets avoids costly wastage of developer time and makes testing a continuous activity, an integral part of the overall CI/CD cycle.

7. Ensuring developer adoption of Unit Testing

In conclusion, we have seen that unit testing facilitates a bottom-up style of testing, by validating individual parts of a program before testing the “sum of its parts“ – an approach that shifts defects left and reduces effort in integration testing later.   

Unit tests act as a kind of “living documentation” of the system, enabling an rapid understanding of a unit’s interface. In the case of test-driven development they can even take the place of formal design. 

Yet unit testing must be done in conjunction with other forms of testing as it not catch system-level or functional errors and also non-functional aspects such as performance.   

The principal challenge with unit testing is the creation of realistic and useful tests, to establish the relevant initial conditions that mirror a normal application execution.  Most importantly, unit tests must be maintained in parallel with the application change process itself, to ensure that impacted tests are kept up to date and are executed for every dependent code change. To make the maintenance of unit tests easy, it has to be easy to re-identify new input and output fields as the application evolves. If not, unit tests can become as buggy as the code they are expected to test! 

To overcome these challenges unit testing must be highly automated and made an integral part of the continuous delivery cycle: 

• Auto-discovery of inputs and outputs to automate the creation of realistic/useful tests 
• Integration of unit testing into the version control process  
• Auto-execution of tests as continuous part of the CI/CD flow 
• Re-use of unit test assets for each program change 

• Dependency-driven maintenance/update of unit tests  

In short, unit testing is a highly beneficial but rigorous discipline, requiring maximal automation for adoption in the long term !