Agile software development using DevOps workflows

by Florian Haßler

Developing and publishing software can be costly. Manual testing, integration and publishing steps take a lot of time and are prone to errors. Therefore, in the past, showstoppers often appeared very late in the course of the project and forced everyone involved to take a few steps back. To avoid this problem, a CI/CD DevOps workflow can be used.

Table of contents

Workflow
Experience with pipelines
CI – Continuous Integration
CD – Continuous Delivery
CD – Continuous Deployment
CI – Continuous Improvement
Benefits
Author

Workflow

The developers select a ticket that has been recorded and finally specified by the customer. They start writing tests and subsequently cast the feature into code (Agile & Test-Driven: Focus on the customer). All this happens far away from the active code base of the application in a separate code area. Before the feature is integrated into the active code base after all acceptance criteria have been met, it is put through its paces in an automated process. Only after it has been tested is it released to the test system and then to the production system. Why automated?

In the automation pipelines, tests can be defined once and thus the associated requirements can always be ensured without human intervention. Containerization additionally helps to make the tests reproducible. Publishing is also automated. A few minutes after implementation, a new feature is already visible on our customers‘ test system and available for user tests. Thanks to automated tests, programming errors rarely reach the test system. This means that customers do not have to deal with these errors and can concentrate on the content of the features. This article goes on to describe what the abbreviations CI and CD stand for at the developers of RISC Software GmbH. The four strategies presented here greatly simplify the everyday work for both developers and users.

Experience with pipelines

There are several platforms for implementing such automation pipelines. At RISC Software GmbH, we have experience with Concourse, Jenkins and GitLab. Especially for the web development teams, we found that GitLab integrated best with their workflows.

CI – Continuous Integration

One idea of Continuous Integration is to integrate new features as quickly and as often as possible in an automated way. This is about short feedback loops. They help developers to identify and fix problems before the context is changed (i.e. the next task is started). The key question is: Can the developed feature be integrated into the existing application without having a negative impact on existing functions?

In simple terms, developers submit their changes to our source code management (SCM) system and the CI pipeline starts working. When finished, it notifies developers of success or failure. Depending on the type of project, different tools are used in the integration pipeline.

It almost always starts with running test suites according to their position in the test pyramid (according to Mike Cohn). Unit tests form the basis of the test pyramid. The smallest possible behavioral units are tested. They ensure that the code works as expected. Integration tests are then used to ensure that the interaction between the various parts of the application or with external components (e.g. databases) also works. If a test fails, the test pipeline is aborted and the dev team receives a notification. Almost always, the last step in the RISC-Software-GmbH CI pipelines is a static code analysis by an appropriate tool. This helps to identify vulnerabilities and inconsistencies with common development practices.

The tool used at RISC Software GmbH contains many built-in rules for different programming languages. The result of the analysis is an evaluation of the technical quality of the source code. It also outputs the number of errors, vulnerabilities, test coverage, number of lines of code written and much more. The developers receive an automated evaluation of their code.

CD – Continuous Delivery

Once the CI pipeline has been successfully run and the report from the code analysis tool is satisfactory, the continuous delivery pipeline is activated manually. It pursues the goal of executing the publishing process in an automated, fast and reliable manner. The artifacts built in the CI pipeline are collected and published to the test system. This publishing step can be simple, copying only source code to a server, or more complex. The target system and the location of the target system determine the complexity.

An advantage that is often mentioned in connection with Continuous Delivery is that the application can be published at any time. There is no time when there is code in the active code base of the version control system that is not functional. If a customer reports a problem, the team can respond promptly and release a new version. Another advantage is, of course, that classic releases are no longer necessary, but feature by feature can be tested and accepted directly by customers.

CD – Continuous Deployment

Continuous Deployment takes the steps of Continuous Integration and Continuous Delivery one step further. Here, all changes that successfully pass through the CI pipeline are also released immediately. This process is fully automated and only a failed integration step prevents the changes from being transferred to the test and subsequent production system. This works if there is a high level of trust in the developers, the Continuous Integration and the Continuous Delivery process. Continuous Deployment has its place mainly in large-scale product development business; in classic project business, the necessary prerequisites are often not economically viable to implement.

CI – Continuous Improvement

The figure below shows the notification that hit some dev teams the morning after the vulnerability in the Java logging library Log4j became known. The security checks of the libraries used run automatically every night to provide the applications we develop with patches as soon as security vulnerabilities become known. They also help to regularly update the software libraries used in applications developed by RISC Software GmbH, thus keeping them up-to-date. Improvements are implemented incrementally, evaluated, and then further improved. The result is leaner workflows and thus more time for the implementation of new features.

Through continuous integration, delivery, deployment, and improvement, RISC Software GmbH has gained the ability to release frequently without compromising quality.

The benefits of our DevOps workflow for customers:

Minimized risk of error (and errors can be corrected more quickly)
Shorter response times (customer feedback can be incorporated more quickly)
Lower costs for manual testing (more features for the same amount of money)
Progress in the development process is visible
Security gaps are quickly detected and eliminated
Components, frameworks and libraries are updated more efficiently

Info

Artefacts

Artefacts are the files created by the build process, e.g. Docker images, ZIP files, logs.

Acceptance criteria

Acceptance criteria describe the conditions under which a requirement implemented by the development team should be considered as fully developed.

Feature

A feature is a specific function in an application (e.g. user administration, login, etc.).

Pipeline

A pipeline in software development stands for the sum of all activities that lead from the idea to the operation of this idea.

Source-Code-Management-System

Source code management system (SCM) is a system used to record changes to the programme code. The version currently in use is mapped in the main branch. Developers have the option of creating branches (like a tree) for features.