No one wants to break the build. Agile development and build automation – from continuous integration (CI) to full continuous delivery workflow – enable development teams to produce a production-ready version of the application at will. However, frequent builds, often running in parallel, are intentionally interrupted when a validation step fails. Rolling back the code and relaunching the build pipeline allows the team to continue developing, but too-frequent rollbacks or defects affecting critical functionality will cause delays and impact the team’s productivity overall.
Production defects are costly
Production defects are costly. Identifying a defect in production costs about 30 times more than the same defect fixed in development and takes about five times longer to resolve. Certain industries – such as embedded software, medical devices, and high-value transactions – experience even higher costs and reputation damage from bugs. Fixing defects in testing costs about 10 times more than development and takes only about twice the time it would have taken before checking in the code. In waterfall development, it would not present a substantial difference; however, in the continuous integration assembly line methodology, defects found in testing still break the build.
Left-shifting defect detection
Left-shifting defect detection, from testing and production to developers, is not a new concept. It just becomes more critical as software powers more mission-critical systems, IoT devices, and backend enterprise operations, driving up the cost and impact of production defects. Left-shifting optimizes continuous delivery workflows by reducing build pipeline breakage and allowing developers, the primary expense for a development organization, to spend less of their time on diagnosing problems.
High unit test coverage and solid coding standards are realistically a prerequisite to any effective build automation. They force more discipline and create awareness to quality, but there are also other tools and techniques that left-shift more defect detection responsibility to the individual developer:
Desktop static code analysis (SCA)
Static code analysis automatically finds potential defects or security vulnerabilities in the source code prior to running the application. SCA tools can be light and look primarily at code syntax, or more sophisticated, examining the complex application execution paths. Industries such as automotive, healthcare, and aerospace mandate the use of such tools in the testing and validation phase. Integrating SCA at the build or CI process improves quality and security. True left-shifting, however, requires wide adoption of SCA at the developer’s desktop, scanning and cleaning the code prior to checking in rather than waiting for the build to fail. Klocwork SCA, for example, combines both desktop and central build\CI time analysis capabilities.
Use code frameworks
Using code frameworks, code components, or libraries, commercial and open source reduce the volume of custom code developed and therefore eliminate defects making their way into the build. Standard “plumbing” tasks – such as complex UI elements, math and analytics algorithms, data mapping, networking, and so on – can be handled by code libraries while developers focus their expensive time on true business logic code. Properly-tested frameworks, supported commercially or by a vibrant community, proactively left-shift problem detection by not introducing “plumbing” code defects to begin with.
Developer side application performance management (APM)
Application performance management solutions provide production performance and failure alerting, including analytics designed primarily for production. It’s unusual not to have comprehensive monitoring in place for production applications. Plenty of commercial and open source solutions are available for different cloud and on-premises environments; however, they are not built with developers in mind. Lightweight APM tools designed specifically for developers left-shift performance problems and error detection to the developer desktop. Desktop APM – such as XRebel APM for Java and Zend Server developer edition or Z-Ray for PHP – allows developers to proactively optimize code before it enters the integration phase.
Adopting standard, automatically-generated application stacks and virtualization or containerization that match the production environments is another practice that left-shifts a class of errors to the developer’s responsibility by not introducing them to begin with. In the cloud or on-premises, standard application stacks reduce chances of configuration and environment mismatch issues from making their way into the build.
No one wants to break the build
Production defects are costly. Developers don’t want to break the build. Every small shift to the left has a substantial impact.