Stackify is now BMC. Read theBlog

CI/CD: What It Is and What You Need to Know

By: Christian Melendez
  |  September 16, 2024
CI/CD: What It Is and What You Need to Know

We’re on the verge of something here, people.

A growing number of companies are shipping software in minutes.

Yeah, you read that right. Minutes. Not hours, not weeks, months, or longer. Minutes.

Often, teams struggle to ship software into the customer’s hands due to lack of consistency and excessive manual labor. Continuous integration (CI) and continuous delivery (CD) deliver software to a production environment with speed, safety, and reliability.

In today’s post, I’ll introduce these concepts, show you how to get it right, and identify what’s important. Then, I’ll include a list of tools commonly used to implement CICD. When you finish reading, you’ll have a better understanding not only of all the benefits that these practices bring, but also the challenges you might encounter.

Simple enough?

Let’s talk CICD.

Continuous Integration

With continuous integration (CI), a developer practices integrating the code changes continuously with the rest of the team. The integration happens after a “git push,” usually to a main branch—more on this later. Then, on a dedicated server, an automated process builds the application and runs a set of tests to confirm that the newest code integrates with what’s currently in the main branch.

System quality—not just velocity—is important.

If you’re doing CI and for some reason the integration fails, that means the broken build becomes the highest priority to fix before continuing to add more features. System quality—not just velocity—is important. CI works in three simple stages: push, test, and fix.

Push to Main Every Day

This CI practice is about pushing code into the main branch, because that’s the branch that’s going to be used to release software.

When you practice CI, it doesn’t mean you’ll no longer use branches. You still will. The only difference is that because you amplify feedback when you integrate code continually, branches become temporary. A branch might live only for the day; then it’s integrated into the main branch.

But what about incomplete changes? Well, you can integrate incomplete changes by using feature flags.

Feature flags are an if condition determining whether to run the new code or not. If a change isn’t complete yet, the flag is off by default. That way, when you integrate the code, the rest of the team has a chance to review it. The same technique applies if the new code has bugs.

feature flags

Rely on Automated Reliable Tests

To validate each time a developer integrates new code, CI relies on an automated and reliable suite of tests.

Include as many tests as possible, but also keep the CI loop as short as possible. You might have to lean heavily on unit testing and lightly on integration testing.

Lastly, when you spot a bug in production, create a test case and include it in the CI loop. You can easily confirm that it’s reliable: If the flag for the bug fix is off and the build breaks; if the flag for the bug fix is on, the build works.

Prioritize Fixing a Broken Build

When the build is broken, fixing it should be the priority for the team. Every time a build is broken, no one else can continue pushing code changes. Everyone on the team is aware of the issues happening.

Besides, whoever breaks the build will try to fix the problem, and if help is needed, then the rest of the team is there. In software, if the build can’t be fixed within minutes, the team should decide if they’ll remove the code or turn the feature flag off. Having a “green” build is not one person’s problem anymore.

Continuous Delivery

Let me use the definition for continuous delivery (CD) from Jez Humble’s site:

Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes and experiments—into production, or into the hands of users, safely and quickly in a sustainable way. We achieve all this by ensuring our code is always in a deployable state, even in the face of teams of thousands of developers making changes on a daily basis.

CI’s mission is then to move those artifacts throughout all the different environments of an organization’s development life cycle. What’s critical in CD is that it’ll always deploy the same artifact in all environments. Therefore, a build in CI happens only once and not for each environment. The artifact produced will work with placeholders or environment variables for the build-once approach to work.

Continuous delivery

The following are the vital principles for continuous delivery. When you put them into practice, they’ll help you to deliver software safely and quickly in a sustainable way, as per the CD definition I included above.

Frequent Small Deployments

Usually, every time a deployment happens, the application’s stability is at risk. Therefore, we tend to distance deployments from each other. But the problem with that approach is that we end up accumulating many changes. Chances are that one of those changes might have problems, forcing us to roll back the other changes that were working. Been there, done that.

What CD fosters—with the combination of CI, which implies quality is built in—is that we should work in small batches.

Automation with a Human Touch

Machines are perfect candidates for repetitive tasks. Automation is a critical principle in CD because it helps to increase the sustainability of the process.

Be cautious, though. Once you introduce automation, you still need people’s minds to improve the process. You can achieve automation with the help of practices like infrastructure-as-codepipeline-as-code, or configuration management.

CD will help to produce one-click deployments that can be triggered on demand.

Always Improving

Are you currently compiling the application from a developer’s machine? Start by using a dedicated server for CI, and build the application in a different server. Are you manually copying and pasting application artifacts? Start by automating the copying/pasting process. It doesn’t matter if you’ve added small improvements or if you’re happy with how advanced the process is. There’s always going to be something to improve. CD doesn’t have an end date.

Shared Responsibility Model

CD produces happier teams because now the deployment pipeline isn’t just an operations problem. Operations will seek ways to help developers build software with quality. And they’ll do it not just by coaching, but also by providing all the necessary tools that a developer might need to understand problems better. A team won’t optimize the process just for them, but it will add improvements that help the organization deliver more value to customers. That’s instead of the traditional approach of rewarding developers for how fast they ship, or rewarding operations for how reliable the system is. If that’s what you’re doing, these two teams will have different goals, and that won’t work.

Everyone in the team is responsible for creating a safer, quicker, and deterministic delivery pipeline continuously.

The Benefits of CI/CD

We’ve already covered a bit about the benefits of CI/CD throughout the post, but I figured it’d make sense to list the main ones in a dedicated section.

Without further ado, here’s a non-exhaustive list of CI/CD benefits:

  • Fewer bugs in production. Since the CI/CD pipeline builds and tests the code after each push to the main branch, fewer bugs make it to production.
  • Shorter lead time. CI/CD ensures software is delivered to the customer at shorter intervals, with quality and a sustainable pace.
  • Spread of software development knowledge. Since all developers quickly and often integrate their code changes, other developers can read and learn from those changes and knowledge is spread throughout the team.
  • Fewer integration conflicts. This is the very genesis of continuous integration: to avoid the once-feared “integration hell” that resulted from long-lasting isolation between developers.
  • Easier maintenance and updates. CI/CD ensures the customer gets fixes and updates to their applications earlier.
  • Better collaboration. With the shared responsibility model, CI/CD encourages collaboration between members of a team, and also between different teams within the organization.

Tools for CI/CD

CI/CD is mainly a cultural shift, but tools can help. Here’s a list of common tools that you can start using today.

  • Jenkins—a free, open-source, Java-based tool that gives you a lot of flexibility.
  • Azure Pipelines—a Microsoft product free for up to five users and open-source projects.
  • Cloud Build—the managed service offering from Google Cloud Platform.
  • Travis CI—a popular tool for GitHub open-source projects that offers a hosted or self-hosted solution.
  • GitLab CI—a free tool from GitLab that can also integrate with other tools via the API.
  • CircleCI—a tool that’s popular for GitHub projects and has a hosted and self-hosted solution. You can start for free.
  • CodeShip—a self-hosted-only solution. You can start with the free version, but it’s a paid tool.
  • Jenkins—can also be used for CD with its pipeline as codeAnsible, or Terraform plugins.
  • Spinnaker—gaining popularity, and it’s the tool that Netflix uses to do releases in a CD way.
  • GoCD—the ThoughtWorks offering that applies the principles I’ve discussed in this post.

This list fails to capture how many tools are actually out there. But the purpose is only to give you a few options. You can use whatever tool works best for you.

A Typical CI/CD Workflow

Next, let me try to assemble all of the concepts I’ve discussed in this post with a traditional but straightforward workflow for a CI/CD pipeline using a Microsoft stack. Each piece can be interchangeable with other tools, platforms, and languages.

What’s important is the workflow, which you can see illustrated here. You’ll definitely want to take a look at the illustration so the summary below makes sense.

In summary, here’s what it shows:

  1. An engineer codes application changes using Visual Studio.
  2. When the code is ready for integration, it’s pushed to a Git repository in Azure Repos.
  3. CI automatically triggers the execution of test cases that will confirm that the code is available for release.
  4. In Azure Pipelines, the release pipeline triggers automatically to deploy the artifacts produced in the CI stage.
  5. An artifact is released into the Azure Web App—let’s say to a development environment.
  6. Application Insights collects information from the site to provide feedback to the team.
  7. The team uses the information available after a release to know the status and impact of the latest version.
  8. Any new feature or bug fix is added and prioritized into the backlog.

When a release to another environment like test or production happens, steps 1-4 won’t need to be run again.

CI/CD: Challenges and Pitfalls

As you’ve seen, CI/CD brings many benefits to the table—once you get it right. It’d be disingenuous of my part, though, if I said adopting CI/CD is a walk in the park. It’s not.

And that’s especially true if it’s your first attempt. There are obstacles you might face, and there are significant errors you must avoid. So, before wrapping up, let’s cover some implementation challenges and common errors in implementing these great practices.

Implementation Challenges

One of the most important implementation challenges for CI/CD is cultural resistance. As you’ve read multiple times during the post, adopting CI/CD is not only about tools and procedures, but about a cultural shift. You might face resistance from people within your organization that resist change on the basis of “this is the way we’ve always done it.” A suggestion here is to invest in education, training people on the benefits of CI/CD. Even consider workshops, so folks can get a first contact with CI/CD and see that it isn’t rocket science.

Another important challenge that you can also solve with education is a lack in skills for CI/CD. The skill gap can certainly pose a challenge, particularly in the first stages, but there’s really not much you can do besides investing in training for employees.

A final challenge would be analysis paralysis when selecting tools. The sheer number of tools available for CI/CD makes picking one somewhat of a daunting task. Fortunately, there are many, many tools you can use for free, so I’d suggest starting with one of those to learn and implement a minimum viable setup, and then considering other options.

Common Mistakes/Pitfalls

An important common mistake when starting to implement CI/CD is to go for an extremely complicated setup without the need for it. An overly complicated configuration will slow down your pipeline, discouraging people from pushing often to the main branch, which is the very opposite of what you want to accomplish!

Also, not taking enough care of your test suite. The test suite is the heart of your CI/CD ecosystem: ensure it has a good coverage of your application, that it’s reliable, easily maintainable, and runs fast.

Finally, a serious mistake when implementing CI/CD is failing to take database migrations into account. Database migrations are always a pain point in deployments; a failed migration can result in outages and data loss, generating financial and reputational damage for the company. The recommendation here is also education: research about how to handle database migrations in CI/CD in a safe, reliable, and reversible way, without causing downtime and loss of data.

Ship Software Continuously

CI/CD helps teams to be more productive when shipping software with quality built in. But the road to having one-click deployments that you can produce on demand is not an easy one. That’s mainly because even though there are powerful tools that will help you to achieve CI/CD more effectively, CI/CD requires a cultural change: a mindset that every person in the team needs to understand very well.

CI/CD helps teams to be more productive when shipping software with quality built in.

I’d recommend that you continue learning with books like Continuous Delivery from Jez Humble. Try to focus more on how to adopt the principles and practices of CI/CD, rather than how or which tools to use. Lastly, CI/CD implementation doesn’t have an end date, like a project, where once you’re at a certain stage you can forget about it. It has to improve continuously. Platforms like Retrace that give you centralized logging and error tracking will help you to improve continuously based on the feedback you receive after each release to production.

Try Stackify’s free code profiler, Prefix, to write better code on your workstation. Prefix works with .NET, Java, PHP, Node.js, Ruby, and Python.

Schedule A Demo

Schedule A Demo

Improve Your Code with Retrace APM

Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.

Learn More

Want to contribute to the Stackify blog?

If you would like to be a guest contributor to the Stackify blog please reach out to [email protected]