By Alan Ashley
We have all heard about the CI/CD pipeline in DevOps. This is the standard phrase. But what is often left out of this is the CT — the Continuous Testing aspect. This could be Code Quality and Security, Unit Testing, or Regression Testing. These tests are the steps needed to “Shift Left” or “Fail Fast” in the pipeline and the use of automation in the CT phase can make the process seamless and reduce the time needed for testing. Granted, your developers may be testing as they develop.
- Are they testing to get the code compiled?
- Are they testing to validate that the changes are not impacting the bigger picture?
- After that, is a QA validation or peer review run? In larger shops, this can be covered by the volume of the team, however in smaller shops, these tests may be skipped just due to resources.
In all cases, automation of the testing is vital to the success of the CI/CT/CD pipeline and with the Arcad testing suite of tools, we fit right in with the use of automation, regardless of the pipeline used.
Let us step back for a minute before we jump all the way through the pipeline. For your organization, define what testing needs to occur and why. As I mentioned earlier, we have code quality, unit testing, and regression testing. Now let’s dive in to identify what you may be missing.
Summary
1. Code Quality and Security
This is the start of the SEC in DevSecOps, but truly you can never really finish when it comes to quality and security of your code. It is forever ongoing. But why? To start, here are a few areas that you need to ask about when it comes to quality and security. As you will see, it’s more than those two topics.
- Developers need to ensure they are following best coding practice for the company and the larger global community. The side benefit to this, any new developer to your organization will immediately notice the similarities in the style of coding. This reduces the onboarding time and improve the ROI for new employees.
- Have you recently decided to or rather forced to quality and security check all your code? Then this process can be slid right into the development cycle and used by developers as they update code or it can be incorporated in a pipeline to kick off a process to review older code to make sure it meets current standards.
- Maybe you have acquired a new company and the code is a mess. How do you go about finding and fixing the flaws? This could be a monumental task.
This is where Arcad’s CodeChecker steps in. Our product comes packaged with nearly 100 rules (such as check for GO TOs, DUMPs, etc.) that have been gathered from industry standards and well as security CVEs (Common Vulnerabilities and Exposures) as it relates to the IBM i. If those rules don’t cover what you need, it has a user-defined option to add even more to meet your company’s needs. For example, this could be the need to validate that the copyright has been added to every source member.
With CodeChecker, it is just a right click away in RDI when using the Arcad DevOps Suite, or it can be a stand-alone install with an eclipse client. Regardless, the functions stay the same, with the server running on IBM i or Windows.
Before moving on to unit testing, just a few more key points on CodeChecker:
- Supports multiple IBM i languages such as the RPG’s, COBOL, CL, and SQL Support
- Pipeline and automation friendly
- Loaded with preconfigure rules to get you start
I know, code quality may not fall under the true definition of testing, although it plays an integral part to ensure you have a solid code foundation but security violation checking does since the impact to Production can be enormous.
2. Unit Testing
Arcad’s iUnit for unit testing again slides right into the Shift Left, Fail Fast methodology. This allows your developers to test the modules and procedures without having completed the entire program. So, whether you code in RPG, COBOL, or even CL, iUnit has you covered. With its automatic discovery of simple or complex parameters in RPG or its mocking capabilities or the inheritance of JUnit features, it has what you need.
Some of the areas that can be tested include Functions, procedures, Programs, service programs as well as SQL procedures. All this function is available right from RDi and coming soon — a deeper integration into the Skipper Plugin found in the DevOps Suite. This new feature will allow right click access to unit testing.
With the inheritance of JUnit features, this is yet another way Arcad encourages new developers from the Open System world to step into the RPG world and the IBM i. Since it has the JUnit feel, it also hooks right into the automation pipeline with Jenkins, if so desired. Personally, I find that being able to test modules on the fly to be the best result for fast results.
3. Regression Testing with Verifier
To round out the testing phase comes regression testing. How do you ensure that previously developed and testing applications still performs the same after a change? This testing usually falls on a QA department or just as often falls on whomever is available.
With Arcad’s Verifier, you can run both manual or automated tests across a single scenario or a campaign to handle multiple areas. The question comes around, which scenario do you run? With Verifier, it builds an ongoing cross-reference repository to determine which scenario to run. If you change program PGM001, any scenario and campaign that runs against that program will be highlighted or selected to run. This feature alone can save you time, which will save you money in the end.
Verifier handles both 5250 and Spooled File comparisons, which is nice, but it digs into the DB2 on IBM i to show file and field level differences. With the DB2 on IBM i review, the use of tools like Selenium can be used to ensure that the data entered and processed at the web entry is what is expected within the backend DB2 database. You can even compare a 5250 entry to a Selenium captured entry just to help identify any errors in the processing aspect of the application.
In addition, it tests for differences in the UI as well. And its supports both batch and interactive jobs. As the pipeline flows closer to deployment, regression testing will be that last line of defense against any defects making it into production, including finding issues because of a modernization project.
4. The final word: CT in the CI/CD
The main objectives to incorporating Continuous Testing in CI/CD workflow:
- “Shift left’ and find issues as early as possible.
- Move from a manual to an automated process to ensure consistency, auditability, and scalability.
- Drive an agile development process.
We have discussed three major areas to the testing pipeline each of which is covered by an Arcad solution. The ‘bonus’ here is these products can function in an existing Arcad DevOps pipeline or be added to existing development pipeline as stand-alone installations.
If you are looking at this and wondering, where do I start, the easiest answer is what need do you have? You don’t need to implement in a specific order. Find the problem area and we can help map a solution. Happy Testing!
And you thought we were done. Did you know you can also pull information from these tools to reflect in a Dashboard? Sometimes this can help see the larger picture. Below are just a few samples of what can be derived from the Arcad VSM Dashboard in conjunction with the testing suite of tools.
Figure 1 – CodeChecker via Dashboard
Figure 2 -Verifier via Jenkins and Dashboard
Now feel free to run off and test, test, and test.
Alan Ashley
Solution Architect, ARCAD Software
Alan has been in support and promotion of the IBM i platform for over 30 years and is the Presales Consultant for DevOps on IBM i role with ARCAD Software. Prior to joining ARCAD Software, he spent many years in multiple roles within IBM from supporting customers through HA to DR to Application promotion to migrations of the IBM i to the cloud. In those roles, he saw first hand the pains many have with Application Lifecycle Management, modernization, and data protection. His passion in those areas fits right in with the ARCAD suite of products.
REQUEST A DEMO
Let’s talk about your project!
Speak with an expert