3 Best Practices For End-To-End Testing

Erik Fogg Developer Tips, Tricks & Resources

As you develop your software, you want it to work well, not only on paper but also as the actual users interact with the application. The way to create quality software that is easy to use and bug-free is with end-to-end testing. 


New call-to-action

What Is End-To-End Testing?

Software testing is riddled with jargon, so our natural starting point would be to flesh out a clear cut definition of end-to-end testing (E2E testing). Simply put, E2E testing is a method of testing your software from the start to the end, exactly how the actual users will be interacting with it. This would include many tiny movements that your user would make, but each one is important. If your software glitches during the user’s interaction, their experience will be more harmful than positive. 

In reality, software should be tested on many levels. First, you would do a unit test, where the programmer writes a minor routine to try. This gives you speedy feedback to make sure that certain features are correctly implemented. Then, following the unit test, you would run an integration test to see how two or more components are used together. 

Although these tests are valuable and provide feedback, there can still be bugs. That’s why you need E2E testing. The unit and integration tests don’t test the UI. That’s where E2E testing comes in. 

Benefits of End-To-End Testing

There are so many benefits to doing E2E testing. For developers, they have the advantage of most of the testing being delegated to someone else, freeing up their time for other projects. For the testers, it allows them to avoid problems in the software. Finally, managers get valuable insight into how a software failure would impact the user. 

Aside from those, these other benefits are worth mentioning:

  • Expands test coverage
  • Ensures correctness
  • Reduces time to market
  • Reduces cost
  • Detects bugs before production
  • Avoids failures

How End-To-End Testing Works

There are three main focus areas when explaining how E2E testing works: methods, lifecycle, and metrics. 

Methods

There are two main methods for E2E testing: horizontal and vertical. Though they both can have a positive result, they are helpful in different ways, and it’s essential to understand the differences. 

Horizontal

A horizontal E2E test will consider the application as a whole, meaning the entire application across multiple systems. This type of E2E testing requires that you have a well-defined workflow and established, defined test environments. It would be best if you planned to set these up ahead of time. 

Horizontal E2E testing essentially includes verifying individual workflow events from start to finish, testing multiple different systems simultaneously. 

As an example, let’s look at an e-commerce application. For a horizontal E2E test, the tester might sign into the user account and browse through the company’s products. Then they would add a few products to the cart and attempt to check out. 

The usefulness depends on the programmers and developers creating tests that focus on the end-user’s perspective. The best part will help alleviate issues in the workflow before the application goes to production. 

Vertical

While a horizontal E2E test takes the application as a whole, the vertical E2E test breaks down the application’s structure into different layers. Each layer has to be individually tested and then analyzed in a hierarchical order. Thus, rather than thinking of the application in terms of workflow, a vertical E2E looks at these layers. 

The layers – which could include UI, database cells, or API requirements – are each tested from the bottom up. They are kept separate from the rest of the layers and tested in isolation before moving to the next layer. Though these are less common than horizontal E2E, it’s still useful in specific scenarios. This is particularly true when there are layers that don’t have a UI or when you have safety-critical software. 

Lifecycle of E2E Testing

Understanding the lifecycle of E2E testing is a great way to understand E2E testing and its usefulness. Four main parts make up the lifecycle of E2E testing: 

  • Test Planning: During this phase, you will identify the critical tasks, associated schedules, and resources needed to complete your E2E testing. 
  • Test Design: As you design your E2E test, you would identify the test specifications, then generate the test case. You would also want to perform a risk analysis and usage analysis before scheduling the tests. 
  • Test Execution: Finally, you execute the test cases. As you do this, you will need to document the results. 
  • Results Analysis: After the tests are complete, analyze the results. You should also evaluate the testing as a whole and perform additional testing if needed. 

Metrics

A considerable part of testing is evaluation is analyzing the results. There are specific metrics to watch:

  • Test Case Preparation Status: This determines the specific position of your test cases under preparation compared to your planned test cases. 
  • Test Progress Tracking: Track this metric weekly. It will give you the details of your test completion percentage, including those passed or failed, executed or unexecuted, and valid or invalid. 
  • Defects Status and Details: This metric gives you a weekly percentage of your open and closed defects. It will also provide you with insight into their severity and priority. 
  • Environmental Availability: This is the actual number of operational hours and hours scheduled every day for testing. 

Best Practices for End-To-End Testing

Here are the most crucial aspects to consider when performing End-to-End testing.

1. Deploy your application properly. It doesn’t help to do a ‘hackjob’ of deploying your software with the idea in mind that you will simply fix everything after testing. Your E2E testing should be there to improve an application that is already well thought out, not fill in all the obvious blanks. So before even considering testing, either, ensure that you are well versed with your host platform and the code that you have written to build everything, or deploy instantly using a specialist cloud hosting service.

2. Do E2E testing after all the other stages of testing have already been completed. That means you should wait until all the individual components have been tested. This will ensure that the system is working and all the unique pieces are connected together as they should. 

3. As you focus on your E2E testing, remember that at its core, it’s testing what the end-user would experience. So get in the mind of your end-user, and you’ll have a successful E2E test. 

4. Once it’s time for E2E testing, you’ll likely follow a specific process. Therefore, becoming familiar with that process is critical to having successful E2E testing. 

The Process

A clear process is not only a time saver but also the best way to ensure the thoroughness of your testing. This is the most common way of doing E2E testing:

  1. Identify the test scenarios: Brainstorm with the business owner, developers, testers, managers, everyone about the different scenarios. These will be the scenarios that mimic what an actual user would do. 
  2. Use scenarios to map out steps: Once you have your scenarios, use those to map out the steps to complete each scenario and identify the results you expect. 
  3. Automate your tests: Automation will give you the most accuracy and speed. 
  4. Test cases in your CI process: This will provide your developers with highly valuable feedback related to each scenario. 

Once you adjust the practice to what works best and you establish a specific process you follow every time for your E2E testing, you’ll speed up your testing time and be more efficient.

About Erik Fogg

Erik is the MIT-educated COO and Co-Founder of ProdPerfect. He loves unleashing the potential of the great folks in the world and loves helping make decisions with facts. A couple years ago he co-founded ProdPerfect, where he helps his team grow personally and improve their ability to help people solve QA problems. He also helps customers use actual live data--instead of educated guesses--when deciding what tests to write and maintain. In his spare time he podcasts and writes books about making fact-based decisions in business and politics.