When you hear the term “software testing,” do you think about one particular type of test — such as functional testing or regression testing — or do you immediately start visualizing the complex, interconnected web of test types and techniques that comprise the broad world of software testing?
Most experienced developers understand that software testing isn’t a singular approach, although, in the broadest sense, it refers to a collection of tests and evaluations that aim to determine whether a software application works as it should and if it can be expected to continue working as it should in real-world use scenarios. Basically, software testing aims to ensure that all the gears are churning smoothly and work together like a well-oiled machine.
That said, there are a variety of approaches to software testing, all of which are equally important in reaching a realistic conclusion to the pressing questions facing developers and testers:
Still, it’s not a simple matter of running a few tests and getting the green light. There’s a process to thorough software testing, which entails writing appropriate test cases, ensuring that you’re covering the right features and functions, addressing user experience concerns, deciding what to automate and what to test manually, and so forth.
We’ve covered many different types of software testing in our recent guide to software testing, as well as in many individual posts (check out our testing archives here). Beyond knowing the ins and outs of software testing, it’s helpful to learn from those who have traveled the path before you to learn from their mistakes and leverage the tips and tricks they’ve learned along the way (and graciously decided to share with the development world). That’s why we rounded up this list of 101 software testing tools.
The list features tips and insights from experts on many of the less black-and-white aspects of testing. Such as considerations for choosing the right tests, creating a testing culture that sets the stage for successful testing among teams, prepping for tests, testing with greater efficiency, and other important insights to streamline your testing process and get better results in less time and, often, at a more affordable cost.
Click on a link below to jump to tips to a particular section:
1. Don’t treat quality assurance as the final development phase. “Quality assurance is not the last link in the development process. It is one step in the ongoing process of agile software development. Testing takes place in each iteration before the development components are implemented. Accordingly, software testing needs to be integrated as a regular and ongoing element in the everyday development process.” – Lauma Fey, 10 Software Testing Tips for Quality Assurance in Software Development, AOE; Twitter: @aoepeople
2. Encourage clarity in bug reporting. “Reporting bugs and requesting more information can create unnecessary overhead. A good bug report can save time by avoiding miscommunication or the need for additional communication. Similarly, a bad bug report can lead to a quick dismissal by a developer. Both of these can create problems.
3. Treat testing like a team effort. “Testing is a team effort. You’ll find keeping everyone in the loop from the beginning will save an enormous amount of time down the line.
“When you expose testers to a greater amount of the project, they will feel much more comfortable and confident in what their goals should be. A tester is only as efficient as their team.
“Your goal is to make sure everyone involved in the project has a solid understanding of the application. When everyone understands what the application entails, testers can effectively cover the test cases.
“Communicate with the test lead or manager to allow testers to be involved in the decision-making meetings. Giving testers access to early knowledge will allow them to prepare early test environments. This will avoid any unforeseen issues, preventing any delays or risks while also being cost-effective.” – Willie Tran, 7 Easy Ways To Be An Efficient Software Tester, Testlio; Twitter: @testlio
4. Use tools to make testing easy. “Most technical leads will be familiar with the challenge of getting developers into the habit of making code testable. Therefore, top of your list of objectives should be ‘ease of use.’ Tests should be easy to write, and more importantly, trivially easy to run, by your development team. Ideally, all developers should be able to run all tests, in a single click, right from in their IDE. No excuses!” – Adam Croxen, Mobile automation testing tips, App Developer Magazine; Twitter: @AppDeveloperMag
5. Find your “good enough” threshold. “Everyone wants perfect software, but budget constraints, business priorities, and resource capacity often make ‘perfect’ an impossible goal. But if perfection isn’t your goal, what is? Recognize that the goal of testing is to mitigate risk, not necessarily eliminate it. Your applications don’t need to be perfect — but they do need to support your business processes in time to leverage new opportunities without exposing companies to unnecessary or untenable risk. Therefore, your definition of quality may vary by application. As you initiate a project, get the right roles involved to ask the right questions: What constitutes perfect versus good enough versus unacceptable?
6. Your user documentation should be tested, too. “User manuals are indivisible from software. There’s no software simple to the point where it doesn’t need a user guide. End users are people who can fall under certain categories and be united by the notion of target audience, but, nevertheless, they are still just a bunch of unique human beings. So, some functionality that is clear to one person is rocket science to another. This proves two points: yes, we all need technical documentation so our product is used properly and, yes, this documentation should be approached from many angles and tested thoroughly to be understood by everyone.” – ClickHelp Team, Testing of User Documentation, TestMatick; Twitter: @TestMatick
7. Keep open lines of communication between testing teams. “Opening up the communication lines between the testing teams can do wonders for making the testing smooth. Communications allow the team to compare results and share effective solutions to problems faced during the test. This will also ensure clear assignment of each task. All members of the team should get updated with the current status of the test.” – Tommy Wyher, Top Software testing tips and tricks you should know, uTest; Twitter: @uTest
8. Automation is good, but it doesn’t fix poor test design. “Test design must take into consideration all the areas of testing to be performed, but it should also identify high-risk areas or other specific areas where test automation would add the most value rather than leaving such decisions to be made ad hoc once development is in later stages.” – 10 Tips to Get Started with Automated Testing, Optimus Information; Twitter: @optimusinfo
9. Testing is about reducing risk. “Testing, at its core, is really about reducing risk.
“The goal of testing software is not to find bugs or to make software better. It’s to reduce the risk by proactively finding and helping eliminate problems that would most greatly impact the customer using the software. Impact can happen with the frequency of an error or undesired functionality, or it can be because of the severity of the problem.
“If you had a bug in your accounting software that caused it to freeze up for a second or two whenever a value higher than $1,000 was entered, it would not have a huge impact. However, that would be a high enough frequency to be very annoying to the customer.
“On the other hand, if you had a bug in the accounting software that caused all of the data to become corrupted every 1,000th time the data was saved, that would have a huge impact but at a very low frequency.
“The reason I define software testing in this way is that — as any tester will tell you — you can never find all the bugs or defects in a piece of software and you can never test every possible input into the software (for any non-trivial application).” – John Sonmez, What Software Developers Should Know About Testing and QA, DZone; Twitter: @jsonmez
10. Think outside of the box. “More and more often we have to deal with assuring quality of various IoT developments. They require testers to become real users for some time and try the most unthinkable scenarios. What we recommend is to start thinking out of the box.
“How can a professional manual tester who runs routine tests regularly become more creative? There are some useful pieces of advice that might be of help to any tester:
11. Don’t rely solely on written communication, particularly for virtual teams. “Especially in virtual teams often the only point of interaction between developers and testers are bug tracking system, yet it is especially the written word that causes misunderstandings and leads to pointless extra work. Regular calls and actually talking to each other can work miracles here.” – Andrea, Successful Software Testing – Communication is Everything, Xceptance; Twitter: @Xceptance
12. Develop “rules of thumb” – and document them. “As testers, we often use rules of thumb throughout a project. For example, we sometimes use a total number of expected defects during test planning and then compare actual defects per hour found versus what we would expect, during test execution. Each of these rules of thumb aids us in managing the information we deal with as testers and QA managers.
“It would be nice (and useful) to have a collection of these rules of thumb in one place, each documented with examples.” – Ray Vizzone, Software Testing and Quality Assurance Rules of Thumb, got bugs?
13. Conduct code reviews. “Four eyes see more than two. That’s why you should let other developers review your source code on a regular basis. Pair programming, on the other hand, a technique where two developers write code together for longer periods, isn’t for everyone and is often not needed. But complicated, important or security related code greatly benefits from code reviews and will improve your code quality a lot.” – Dennis Gurock, 12 Practical Tips for Building Bug-Free Software, Gurock Quality Hub; Twitter: @gurock
14. Manage defects in code during development, particularly for complex code. “Rather than rely on traditional QA testing methods, Developers and development managers should also be able to quickly and easily manage the defects in their code – especially where code is complex. This includes prioritizing defects based upon impact and filtering defect information to view only what’s relevant to them. Once the defects have been prioritized, developers should be able to automatically find all of the places the defect exists across projects and code branches – thus minimizing duplication of efforts. Then they should be able to collaborate with other developers to share triage information across distributed teams and geographic boundaries.” – Chris Adlard, Five Tips to Make Software Testing Easier, Database Trends, and Applications; Twitter: @dbtrends
15. Report findings in the context of business value. “Focus on the data that is being communicated back to stakeholders, from your findings as part of testing – the data should be in context of ‘how’ the behavior observed is detrimental to the objective of the feature or application being developed.” – Mush Honda, 9 Steps to Becoming a Great QA Lead, KMS Technology; Twitter: @kmstechnology
16. Engage the end user. “Probably the most important person in the whole process, yet many times we may be tempted to keep them at arm’s length; you should involve the customer actively. Have them give frequent feedback on the product for future improvement and development; software developers who respond quickly to customer feedback are generally more successful.” – 5 Tips for Developing an Effective Software Quality Testing and Assurance Culture, Techno FAQ; Twitter: @Techno_FAQ
17. Always keep learning. “[The] IT field changes; way [faster] than some of us would like.
“If you are not constantly updating your skills, you could get irrelevant, obsolete and outdated. In a world of lay-off paranoia, it is a good idea to rise above it all, gain immunity and feel secure. The best way to do so is to make learning a habit.” – Swati Seela, How Can Testers Embrace Learning and Keep the Spark Alive?, Testing Excellence; Twitter: @TestingExcel
18. Bug summaries must be thorough. “Most customers including your managers, developers, and peers will read the summary first when they review a bug. This is especially true when they have more bugs to review.
“The simple reason is that they don’t have enough time to go into details of every bug, so having a short and concise summary will surely help to grab an idea of what the problem is about and how important it is.
“You can have a short and concise summary by telling exactly what problem you found and in what condition.” – Thanh Huynh, LogiGEAR Magazine; Twitter: @logigear
19. Use Test Maturity Model integration. “The software industry does not operate in a zero-defect environment, and, arguably, it never will. In the face of this truism, numerous techniques to reduce the number and severity of defects in software have been developed, with the ultimate, albeit unobtainable, goal of defect elimination. Such optimistic thinking has led to significant improvements in software quality over the past decade, notwithstanding increased software complexity and customer demands.
“One such defect elimination approach is maturity models. Broadly, these are structures which state where an organization sits on a maturity scale, where its failings lie and what should be done to improve the situation using process improvement frameworks. The archetypal maturity model is the Capability Maturity Model Integration (CMMI)2, in addition to its predecessor, the Capability Maturity Model (CMM).” – Dr. Mark Rice, Test process assessment: climbing the maturity ladder, Software Testing News; Twitter: @testmagazine
20. Always start with a product map. “Early on in the project you should spend some time exploring the software, and try to model the features and requirements of the product. A graphical model (for example, a mind map) can provide a concise, easy-to-understand representation of the product, and the modeling process is likely to help you uncover features that you may not previously have been aware of.” – ChengVoon Tong, Top 3 tips for software testing a mature product, Redgate; Twitter: @redgate
21. Getting testers involved from the start means you can eliminate many errors even before reaching the development stage. “When testers start working on the project from the very beginning, they make sure that many errors are identified and eliminated even before the development phase. Writing test scripts, quality testers assist developers that can later use these scripts for making product creation easier. Thus, involving testers into work at the first stages of development has a range of advantages: helping the team to understand customers’ goals, saving a lot of time, minimizing expenses, and optimizing the approach to testing.” – Mark, Top 7 Tips for Choosing an Outsourcing Software Testing Team, The Merkle; Twitter: @themerklenews
22. Choose flexible test management tools that can adapt to your needs. “No two businesses are the same which might mean a particular tool is best-suited for a situation different to yours. Keeping this in mind, you should look for a test management tool which not only fits your day-to-day testing needs today but should also offer flexibility if your testing approach changes course in the future.” – Sanjay Zalavadia, 5 Most Important Features to Look for in Test Management Tools, Quick Software Testing; Twitter: @quickswtesting
23. Create sample test data if needed. “Depending on your testing environment you may need to CREATE Test Data (Most of the times) or at least identify a suitable test data for your test cases (if the test data is already created).
“Typically test data is created in-sync with the test case it is intended to be used for.
“Test Data can be Generated –
“Typically sample data should be generated before you begin test execution because it is difficult to perform test data management. Since in many testing environments creating test data takes many pre-steps or test environment configurations which are very time-consuming. Also If test data generation is done while you are in test execution phase, you may exceed your testing deadline.” – Tips and Tricks to Generate Test Data, Guru 99; Twitter: @guru99com
24. Aim for stability. “Stability is always important; your tests must always run and spit out the correct results. What good is having a test suite if it’s giving you false positives and negatives?” – John Howard, Tips & Tricks for creating a great Automated Testing Suite, uTest; Twitter: @uTest
25. Make sure developers have the test cases. “It is considered a good practice if tester gives his test cases to the developer to verify that all the important functionalities are developed properly before he releases the application for further testing. It ensures that re-work would be minimum since most important part of the application is taken care by the developer himself.” – Software Testing Tips And Tricks For Testing Any Application, Software Testing Class
26. Follow a proven process for functional testing. “Functional testing reviews each aspect of a piece of software to make sure it works (aka functions) correctly. Quite simply, functional testing looks at what software is supposed to do and makes sure it actually does that. So while functional testing looks at an application’s ability to execute, non-functional testing looks at its overall performance (e.g. by testing scalability, reliability, security, and compatibility).
“When conducting functional tests, you typically need to follow a process that looks something like this:
“Following this method, if the expected results and the actual results match, then you can conclude that the software functions properly and the test has passed. If they do not match (assuming you properly understand what the outcome should have been and used the correct input), then there is an issue with the software.” – Functional Testing Types – 25 Best Practices, Tips & More!, QA Symphony; Twitter: @QASymphony
27. Understand the data flow. “When you know how the data travels inside your application, you are better able to analyze the impact of component failures and security issues. Hence, recognize how the data is used within the application early on in order to report bugs and defects faster.” – Eight Tips to Be More Effective in Agile Software Testing, MSys Technologies; Twitter: @MSys_Tech
28. Write your tests for the correct features to cut your maintenance costs. “What’s the easiest test to maintain? The one you didn’t write.
“UI automation is hard. It’s a difficult domain to work in, the tests are slower to write than other tests, they’re slower to run, and they’re harder to keep working. If you want to keep your maintenance costs as low as possible, carefully consider what you’re writing your UI tests for.
“Focus your UI automation efforts on high-value features. Talk with your stakeholders and product owners. Find out what keeps them awake at night. I’ll bet it’s not whether or not the right shade of gray is applied to your contact form. I’ll bet it’s whether customers’ orders are being billed correctly. I’ll bet it’s whether sensitive privacy or financial information is at risk of being exposed to inappropriate visitors to your site.
“Automate tests around critical business value cases, not around shiny look-and-feel aspects of your application.
“Write your tests for the correct features. It will cut your maintenance costs dramatically.” – 10 Tips on How to Dramatically Reduce Test Maintenance, Telerik; Twitter: @Telerik
29. Channel an attacker for security testing. “Try to get into the mindset of a potential attacker. Just as you try to emulate the end user when software testing, with security testing you want to emulate an attacker. It’s fair to assume that they’ll seek entry via the path of least resistance. Start with the most common methods and attack scenarios. But it’s important to remember that nothing’s off the table because an attacker will do anything that will get them the data they want.” – Simon Hill, 8 tips for security testing web applications, Crowdsourced Testing; Twitter: @crowdsourcingqa
30. For apps, including the device in your testing plan is imperative. “An application that comes packaged on a consumer-grade laptop or notebook for a police squad car will not withstand the rigors of high-speed chases and constant bangs and knocks. Part of the application testing strategy, if you are developing for situations like this, should include the testing of the robustness of the device itself in adverse operating conditions. If you fail to include the device in your test plan, the app might be great — but it might also crash at a critical moment if the end device fails.” – Mary Shacklett, 10 tips for testing apps for the real world, TechRepublic; Twitter: @TechRepublic
31. Test-writing methodologies and concepts must be grasped before using automated tools to “turn the crank.” “When I hear about these new testing tools, I generally view them as new methods to turn the crank in testing. Anything can execute a test plan, after all, there’s no fundamental reason why a human needs to run through a test plan versus a machine. Both are capable of turning the crank. There’s also no fundamental reason why a human needs to write a test plan either. Well, other than machine learning hasn’t gotten that good yet.
“The part many of these tools seem to leave out is how these tests get written. The methodologies and concepts need to be grasped before you start turning the crank. Extracting anything useful from automated testing requires a robust set of test cases. You need to have clear goals – launch the app, poke this set of buttons, get this result. This is true regardless of which method is used.” – Kirk Chambers, QA tips and tricks: Why a clear and robust test plan is essential, Possible Mobile; Twitter: @POSSIBLEmobile
32. Avoid cross-browser variation. “On a new project, you might be tempted to use a slew of emerging browser capabilities. You will often need to use feature detection so that visitors with older browsers get a usable fallback. This means that you need to test the same feature across browsers with different expectations of what is correct.
“It can be easy to get carried away using the latest tech, but a significant part of your audience may be using older, less capable browsers. Sometimes it is better to use a well-established approach for everybody.
“By using newer browser features with inconsistent support, you are deliberately introducing cross–browser variation. We know from the Browser Wars of the 1990s that this comes at a cost.
“Pick the technologies you use with care. Restrict your choice of newer features to those that will have the biggest net benefit to the user.” – Jim Newberry, 31 Ways to Spend Less Time on Manual Cross–Browser Testing, Tinned Fruit; Twitter: @froots101
33. Define entry and exit points. “Understand your software application being tested completely. Here, we do take care of when and how the testing of a particular testing phase will start and end. So, this will help us to decide how and which automation testing framework can be involved for a particular testing stage.” – Manish Verma, Top Software Testing Best Practices and Automation Strategy, Software Testing Mentor; Twitter: @swtmentor
34. Run a pilot project before full-scale automation testing tool adoption. “Generally, a pilot project is kick started by preparing a business case describing the objectives of the project and outlining the methodology how the project shall be carried out. A realistic time plan along with metrics for the determination of the success is an essential part of the business case. For instance, the testing engineer may like to reduce the time to run the regression tests from a week to a day. Actually, applying the ‘don’t be overly optimistic’ rule, it may be better to set a target such as reducing the time for 20% of the tests by a factor of 50%. This may result in a five-day regression test taking four and a half days, but might be a much easier target to hit.
“The pilot project must not be either too short or too long, may be from 2 to 3 months. Subsequent phases of the pilot project could extend this time beyond 3 months but each phase should have measurable objectives. If the pilot stretches for a longer period without significant results, it would cast a shadow of doubts on the viability of the overall test automation. It is better to accrue smaller benefits in the beginning may be in bits and pieces, being less risky rather than gaining much larger benefits that are projected later.” – Importance of doing a Pilot Project before Full Scale Automation Tool Roll Out, Software Testing Genius
35. For testing mobile apps, emulators can be useful, but they can’t perfectly replicate a real-world operating system. “Some tech experts use emulators for application testing. And it is good because an emulator is a powerful tool that makes app testing easier and cheaper. However, emulators lack many features inherent only to real operating systems. Considering Murphy’s Law, any missing feature in the emulator that CAN go wrong in the real environment, WILL go wrong and cause troubles. So, before release, test an application using target OSs on physical mobile devices.
“Moreover, it is necessary to check how your application works on different versions of the target operating system. For example, if your application is intended to work on iOS 9, then try 9.0, 9.1, 9.2, etc.” – Mobile App Testing Tips & Tricks, Skelia; Twitter: @Skelia_company
36. Don’t underestimate the impact of maintenance overhead. “This one is particularly tricky. People often do not realize the cost of maintaining an automated testing infrastructure. If you are writing test scripts for a rapidly changing application, you should gather all the information that you need and then take some time to come up with an estimate for this overhead. Having a solid testing setup really makes the difference here: fixing broken tests is faster when you have clean, readable test scripts with little to no duplicated code. Following a pattern such as PageObject can help you build such a setup.” – Giovanni Rago, Top 5 Mistakes That Can Sabotage a Successful Test Automation Project, SauceLabs; Twitter: @saucelabs
37. Test the instructions. “If a test has too many twists and turns, you’re testing users, not the website. Ask someone else on your team to try the test (not just read the steps) and have them note any unclear or confusing instructions. Run a pilot test and see if you get the intended results.
“Save big picture questions for the summary, when test participants have completed all tasks and have a moment to reflect.
“Tip: Request the same test participants for later rounds. This allows you to test the experience of repeat visitors and find out whether you’ve addressed their feedback.” – 12 Tips for Top Test Results, User Testing; Twitter: @usertesting
38. Consider the complete, end-to-end user journey. “A user journey is a series of steps which represent a scenario in which a user might interact with a system. Typically, a user journey has a starting point, i.e. an entry point into the system, a series of transitions from one state to another and a set of triggers which cause the transitions.
“User journeys can help you identify customer behavior and how the users use the system or how they could potentially use the system.
“When we are creating user journeys, we need to be thinking about:
“The important part
