Top Bug Tracking Tools

Top Bug Tracking Tools: 50 Useful Tools for Tracking and Collaborating on Bugs and Issues for Better Software Development

Angela Stringfellow Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Dealing with code problems is a natural part of the development process. For small projects, you can sometimes get away with local testing and local notes, but as your software keeps growing, it’s a good idea to start thinking about a centralized bug and issue tracking solution that can archive the progress of your development cycle.

Likewise, it will be helpful for everyone else on the team to have an environment where collaboration can happen in real-time. With any of the following bug tracking tools, you can quickly create a separate dashboard that can be used for managing any problems related to code, and its functionality.

Because of the nature of the task — bug/issue tracking — a lot of these platforms also offer general project management functionality so that you can manage your product and/or software from a single place. Note that the following 50 tools are not ranked or rated and are listed in no particular order. Of course, we can’t make a list of bug tracking tools without mentioning our Retrace, which you can find at #50.

1. Stryka

@Plutora

Stryka

Tests are the essence of a successful DevOps team, and Stryka bridges two fundamental aspects of the development process — test management, and test bug reporting, all in a beautifully designed dashboard. Get on top of your test workflow and see how your team can work together on solving issues at hand.

Key Features:

  • Works seamlessly in the cloud, from testing to execution.
  • Globalized login module that lets you manage all projects from one user account.
  • Built using industry-standard frameworks for outstanding user experience.
  • Custom leaderboards to see how teams are performing and what improvements are being made.

Cost: Free 30-Day Trial

2. DoneDone

@GetDoneDone

DoneDone

Bug testing is about simplicity, and DoneDone emphasizes that belief throughout the whole of its platform. Get an actionable workflow that you can use to track customer tickets, and work through bugs with your team as soon as they appear. All in a simple, yet effective, issue tracker.

Key Features:

  • Customer user prefixes that define who is in charge of which ticket.
  • Flexible reports that show how much is getting done on a daily basis.
  • Browser-supported notifications in real-time.
  • Sync your GitHub and Bitbucket accounts for your commits.
  • Public issue tracker for collecting user feedback.

Cost: Starter at $39, Premium at $79

3. Request Tracker

@BestPractical

Request Tracker

Best Practical Solutions built Request Tracker as a means to help development and enterprise teams of all sizes to manage their bug tracking environment in a slightly more flexible way. As an open-source project, it’s self-hosted on your server and is supported by a database back-end.

Key Features:

  • Simple UI focusing on the essentials of issue tracking.
  • Compatible with modern browsers and browsing devices.
  • Email integration that can fetch emails directly from your key business accounts.
  • Custom assets environment for setting up custom tracking workflows.
  • Integrated knowledge base for hosting documentation.

Cost: Open Source

4. BontQ

@BontQ

BontQ

Bontq has been around for the last six years, providing teams with a web-based issue and bug tracking platform, and an integrated project management solution that can help teams to focus on set goals within a single dashboard. Famous for its Desktop Client that can be used to take screenshots or record videos of any particular issues at hand, making it less of a hassle to understand the problem at hand.

Key Features:

  • Custom tracking definitions for bugs, features, notices, etc.,.
  • Global reporting to understand who is doing what and what impact their actions are having.
  • All activity is monitored in real-time using email notifications.
  • Use your Google Docs documents to enhance project management.
  • Chat groups to talk with your team members in real-time.

Cost: Free Trial, Business Plan Starting at $9

5. WebIssues

WebIssues

WebIssues is both an accessible web platform, and also a desktop-based solution for issue tracking and collaboration amongst teams. It’s a universal database of your code workflow that can track, store, and enhance your issue reporting environment. While it’s easy to get going and works out of the box, it is flexible towards customization.

Key Features:

  • Desktop client for Unix, Windows, and OS X systems.
  • Custom web client to access issues using a web dashboard.
  • Custom filters based on public/private requirements for individual tickets.
  • Check existing or upcoming tickets against custom filters to recognize patterns.
  • All communications are behind SSL, and an anonymous environment is also possible.

Cost: Open Source

6. Bugzilla

@Bugzilla

Bugzilla

With nearly two decades of operation time, Bugzilla has stood through the test of time to come out as one of the leading bug tracking (and testing) tools on the market. It is web-based and is aimed for all general purposes of bug and issue management. Bugzilla’s key advantage is an active community that continuously contributes towards the betterment of the codebase and list of features.

Key Features:

  • Complex search algorithm with custom prefixes and filters.
  • Custom bug list formats for calendar apps or code editors.
  • Advanced visual dashboard for reports and charts.
  • Duplicate detection system to avoid submitting the same bug twice.
  • High security precautions to keep intruders at bay.

Cost: Open Source

7. Mantis Bug Tracker

@MantisBT

Mantis Bug Tracker

Mantis Bug Tracker has been kicking bugs in the butt since early 2000’s and is still going strong even today. MantisBT is an open-source bug tracking system to be utilized in a web-based interface. It is typically used to track software bugs, but you’ll see some teams configure the dashboard for issue tracking and project management.

Key Features:

  • Custom plugins system that lets you create custom functionality.
  • Works on any platform with PHP support.
  • Custom monitoring and tagging for specific reports.
  • Create RSS feeds for virtually any live data within the platform.
  • Select from a public or private environment, based on tickets or projects.

Cost: Open Source

8. Axosoft

@Axosoft

Axosoft

Axosoft is a sophisticated project management tool with inbuilt bug tracking solution. It is currently available to be hosted on the cloud, or on your physical location. Most frequently Axosoft is used by teams who are planning their product launch from first to final phase. Axosoft operates within a web browser and has the capabilities to integrate with Microsoft Visual Studio and TortoiseSVN.

Key Features:

  • Release management using inbuilt task manager.
  • Create and manage sprints, including their bugs.
  • Kanban board for visualizing your project timeline.
  • Assign hours (standups) of work for particular tasks of individual team members.
  • Powerful visual dashboard for analyzing your progress.

Cost: Free for 2 Users, Self-Hosted Starting at $250 per User

9. eTraxis

eTraxis

eTraxis is a record tracking system with an ability to set up an unlimited number of fully customizable workflows. eTraxis can be used for tracking almost anything, but the most popular cases are bugs tracker and helpdesk system. It’s acclaimed for its straightforward approach and easily manageable dashboard.

Key Features:

  • Custom templates for creating workflows.
  • User management with flexible permission frameworks.
  • Database supports for PostgreSQL and MySQL.
  • Visualization algorithm for measuring project progress.
  • Comprehensive history reports for events and changes.

Cost: Open-Source, Free for 5 Users, $50 for Unlimited Enterprise

10. BugNET

BugNET

BugNET is a bug/issue tracking solution built using .NET — it can be configured to be used as a project timeline management platform, with the main priorities being the ability to maintain a smooth codebase, an organized code documentation, and software that’s easy to deploy and scale. It is an open-source project.

Key Features:

  • Assign users for individual projects.
  • Custom fields to better index the context of bugs.
  • Easy email notifications using POP3.
  • Issue types to categorize new requests; bugs, features, events, etc.

Cost: Open Source

11. The Bug Genie

@TheBugGenie

The Bug Genie

Getting stuck in an ocean of bugs without a centralized system to manage them all, well, it can get quite messy quick! The Bug Genie from Zegenie Studios is a flexible solution for those teams who need a centralized platform for managing their issues, and also independent projects. Teams can sign up for custom training classes to get everyone up to speed with how to make this solution a part of their daily routine.

Key Features:

  • Create dashboard panels that classify your project roadmap.
  • Projects are based on ‘spaces,’ which let you create custom project directories for individual parts of your software.
  • All project history is recorded in a timeline, and you can access it whenever.
  • Add custom fields to your issue type schemes, use them in searches and integrate them into your workflow.
  • Includes a sophisticated Wiki platform to detail your documentation, tutorials, and other crucial project information.

Cost: Free for 10 Users, $299 for Enterprise (Unlimited)

12. Lighthouse

@LighthouseApp

Lighthouse

The Lighthouse App is an online-based project issue management and tracking software, built with the intention to help teams of any size to monitor the incremental growth of their project, based on resolved bugs and issue at hand. Fully customizable through the use of the Lighthouse API, but can be integrated with simple developer software, too.

Key Features:

  • Intuitive interface optimized for easy navigation between tickets and overall functionality.
  • Custom tagging system to help you organize your tickets in accordance with your routine.
  • Create milestones to keep your team on toes about frequent releases.
  • Native integration with GitHub, Beanstalk, Airbrake, and others.
  • Designed for mobile flexibility so you can stay up to date no matter where you are.

Cost: $25 for Startups, and $100 for Unlimited Projects

13. Zoho Projects

@Zoho

Zoho

Zoho is a fantastic company that has worked hard to release countless of high-quality products for developers, designers, and general technology teams across the vast field of IT. Their Bug Tracker is a complementary module for the Zoho Projects software. Both go hand in hand to provide a seamless experience for bug and issue tracking, while keeping it all underneath a project.

Key Features:

  • Custom filters to monitor issues at hand based on their critical state.
  • Create custom bug management cycles to keep progress steady.
  • Detailed ticket reports that can encourage developers to focus on fixing bugs at hand.
  • Specific and automated notifications for reminding devs to fix a particular bug and/or issue.
  • Improve productivity by exactly knowing which bugs are reproducible, to what level.

Cost: Free for 1 Project, Premium at $50

14. Team Foundation Server

@VisualStudio

Team Foundation Server

TFS is an advanced solution coming from Microsoft that provides DevOps, Engineers, and Software Developers with an extensive source-code management solution, which includes aspects like reporting, project management, build automation, testing and deployment management, and last but not least, a properly defined issue tracking interface.

Key Features:

  • Streamline your process from an idea towards a final application.
  • Works with any modern programming language.
  • Use any Git client of your choice.
  • Integrations of agile tools for increased productivity.
  • Collaborate on code easily with pull requests and code reviews.

Cost: Free for Small Teams, and $30 for 10 Users

15. Unfuddle

@Unfuddle

Unfuddle

Unfuddle’s user interface might remind you a little that of Trello, but don’t let that fool you. While the board’s interface is lovely, the powerful features and functionality are what sets Unfuddle apart from the rest of the crowd. Teams can enjoy a ‘boards’ kind of environment, while having the full support towards the management of modern software and programming languages.

Key Features:

  • Powerful collaboration features to keep everyone involved with the project issues at hand.
  • Seamless Git hosting.
  • Categorize your issues and tasks based on a custom structure.
  • Create as many projects as necessary.
  • Extensive management of issues, possible thanks to a clever interface.

Cost: Free for Personal Projects, $19 for 5 Users

16. DevTrack

@TechExcel

DevTrack

DevTrack from TechExcel is a one-stop solution that will help your team shorten project deadlines, improve the way your project issues are being addressed, and overall promote the quality of products that you’re shopping. DevTrack lets you track the progress of multiple projects, include that of team members and their appropriate clients.

Key Features:

  • Integrated dashboard for connecting your project with anyone involved in the process of building it.
  • Custom SCRUM templates to help you maximize productivity.
  • Linear planning tools to streamline the lifeline of your project.
  • Intuitive defect (bugs and issues) tracking system that integrates with the known history of a product.

Cost: Free Trial

17. BugHerd

@BugHerd

BugHerd

BugHerd takes a modern approach of bug management, and uses the latest available frameworks for creating a bug tracking tool that’s simple, efficient, and produces tangible results in as little time as possible. They call themselves ‘the sticky notes of bug tracking,’ and it’s certainly not far from the truth. It is strictly built as a platform to be used on live websites, so the bugs reported will be in relation to your active production software.

Key Features:

  • Point and click bug reporting widget directly into the browser.
  • Automated screen capture of the particular issue, which is uploaded alongside the actual ticket.
  • Detailed reports of the user submitting the report; browser, selector, resolution, etc.
  • Create and monitor projects to see how your team is performing.
  • Integrates with GitHub, Zapier, Basecamp, and many other leading apps in the field.

Cost: Standard at $29, Deluxe at $180 for 50 Users

18. FogBugz

@FogCreek

FogBugz

FogBugz is an online based solution for project management that has custom support for big and issue tracking, an integrated discussion forums, wiki support for documentations, and even a customer relationship platform to keep up to date with how your customers are doing. Agile, Issue Desk, Wiki and Dev Hub — the additional features, are independent addons that can be enabled or disable for each of your projects.

Key Features:

  • Effortless task management with quick entries for tasks and issues.
  • Create custom structures and workflows based on the department you’re working with.
  • Monitor tickets and tasks that need to be resolved soon, with an estimated time of completion.
  • Smart scheduling takes into account past estimate accuracy.
  • Edit issues in bulk to improve productivity.
  • Fetch GitHub commits.

Cost: Starting at $20 for 5 Users

19. TrackDuck

@TrackDuck

TrackDuck

Not all bug tracking software is going to be the same, that’s why they all exist in the first place, but TrackDuck takes it to another level with an intuitive visual bug reporting solution that integrates directly into your website pages. With TrackDuck clients provide feedback directly on the web site right where they are.

Key Features:

  • All issues have in-depth details of the reported bug, including snapshots, comments, and a direct marking over the issue on the website itself.
  • Client and customers can click on any part of the website to leave a comment.
  • Works with Trello, JIRA, GitHub, Slack, Asana, and other leading web tech software.
  • One-stop visual feedback solution.

Cost: Starting at $9 per month

20. Backlog

@BacklogTool

Backlog

Backlog is a custom-built software solution with a primary focus towards the management of tasks on a daily basis. Using an intuitive drag and drop interface, users can plan their day ahead of time to know exactly what needs to be worked on and why. With a native project management back-end, Backlog can also be tailored for the use of tracking and solving bugs.

Key Features:

  • Issue progress tracking to stay frequent with bug-fixes!
  • An extensive list of attributes to select for each new ticket.
  • Create estimates for all tickets and see how much actual time they took.
  • You can search issue by Status, Category, and Milestone. Plenty of more filters is available.
  • Support for Gantt and Burndown charts for a visual timeline of your project.

Cost: Free for 1 Project, and Premium at $100

21. Aiveo

@YGenDevelopment

Aiveo

YGen Development works on some projects with their clients, but one of their personal flagship products is Aiveo — a flexible project management solutions that are aimed at small business owners, individual teams, freelancers, and medium-sized business teams. Aiveo can take care of planning a project, reporting bugs for it, and the general in-house tasks related to working with your project on a daily basis.

Key Features:

  • Modern UI of the web dashboard makes it easy to access all aspects of your project.
  • Work items act as issue management filters.
  • Is anyone slacking? Are bugs taking longer than usual? Track the time required to solve issues.
  • Custom task board to visualize progress and general status.
  • Manage your project’s requirements, user stories, or goals with objectives.

Cost: Free for 5 Users, Business Starting at $10 per month

22. Fossil

Fossil

Fossil is a VCS (version control), bug and issue tracking platform, and wiki solution all within a single bundle of software. It runs on Unix, OS X, and Windows systems, with in-built capabilities for distributed work environment amongst versions controls, issue tracking, wiki production, and even blogging aspects.

Key Features:

  • Smooth web interface for managing data online.
  • It is self-contained, meaning you only need to download and compile.
  • Easy networking using HTTP(s) and SSH for any communications across the network.
  • The automatic self-check mechanism helps insure project integrity.

Cost: Open Source

23. Phproject

Phproject

Phproject, as the name suggests, is a project management system built with PHP. With a completely effortless setup, you can have your web-based project management system running in no time. Clone the repository from GitHub into your WWW folder, open the page from your browser, input database details and you’re good to go!

Key Features:

  • Database queries kept to a minimum to improve performance.
  • In-built caching system to stop the server from running slow and repetitive queries.
  • The task board displays the projects and tasks in a sprint with a scrum board interface.
  • Keep track of your project history with a concise backlog diagram.

Cost: Open Source

24. Trab

Trab

Trab is yet another website-based feedback gathering solution that collects feedback and user bug reports directly from the pages that the user is viewing the site from, this can include your software pages, too. With the addition of a tiny widget, users can click the button and move the cursor over the area they feel is broken, then specify their findings and submit alongside a snapshot image of that particular page.

Key Features:

  • Modern approach to tracking and fixing bugs in a live environment.
  • Pixel-perfect cursor matching algorithm to narrow down broken parts of the website and/or software.
  • Custom role management to assign devs to specific bugs.
  • Set it up by dropping an HTML code in your footer, that’s it!

Cost: Free for Public Projects, $30 for Agencies

25. Redbooth

@RedboothHQ

Redbooth

Redbooth is an intuitive on-premises project management and collaboration tool for software and developer teams. It’s built not to be constructed for a particular group of people, so anyone and everyone can use it! With more than 500,000 customers — Redbooth is one of the leading task/project management tools out there.

Key Features:

  • Tasks can be assorted by their individual stages.
  • Intuitive interface that reminds you about upcoming tasks.
  • Assign tasks/tickets to users and monitor their activity over time.
  • Custom templates for projects related to clients.
  • Visualize projects at a glance in a beautiful Gantt chart.

Cost: Pro at $5 per user, and Business at $15 per user

26. DebugMe

@DebugMe_eu

Debug Me

Who needs complicated bug tracking solutions when you can make a permanent switch to visual feedback, given that your application/software is only web-based. DebugMe is yet another excellent choice for those who wish to enable bug and issue tracking directly into the browser/website experience. With native integrations amongst JIRA, Trello, Redmine, Trello and other leading platforms, you’ll instantly increase the productivity of any bug reporting that happens on your website. 

Key Features:

  • Third-party integration with Mantis, Asana, Slack, GitHub, Evernote, and other similar platforms.
  • Authentic toolbar for adding feedback on the given website page.
  • Kanban board environment to keep everyone updated about progress.
  • All reports include in-depth technical information of that page/situation.
  • Create groups and invites others to join your project.

Cost: Free Trial

27. Trello

@Trello

Trello

Trello is a collaborative platform for creating card-style boards and lists for managing projects, or the general workflow of your life. The situations in which Trello can be useful are truly limitless. And bug tracking is certainly one of the things that Trello is great for. Teams can create custom boards for a timeline of bug reporting and the process of fixing them while having the ability to invite more people to participate, and archive all work that’s being done on each issue/bug.

Key Features:

  • Beautiful visual environment for managing virtually anything.
  • Creative, but simple to understand, filters and categorizing options to create a broader outline of your projects.
  • Integrates with your favorite apps that you are using on a daily basis already.
  • Trello is a mobile-friendly platform so that you can stay in touch with your team from wherever!

Cost: Free and $10 for Business Class

28. Redmine

Redmine

Redmine is an open-source project written using Ruby on Rails. It is multiplatform and multi-database oriented. Issues in Redmine can be tasks, ideas, software bugs or support tickets. Redmine is highly configurable so that you can create custom issue types for specific situations. At its core, it’s a project management platform and has a bustling community of developers and software teams that continue to support the project on a daily basis.

Key Features:

  • Create and manage multiple projects.
  • Access is managed by user roles for custom permissions.
  • In-built calendar and Gantt chart reporting.
  • Custom forums and wiki pages for individual projects.
  • State of the art issue tracking solution.

Cost: Open Source

29. SpiraTest

@Inflectra

SpiraTest

SpiraTest integrates directly with the main workflow of your app production structure, it’s both a testing and management solution, as it is a bug tracking platform that will complement those tasks by themselves. SpiraTest is being used by customers in every industry to manage their Quality Assurance process.

Key Features:

  • Bug tracking tools that manage your testing environment.
  • Create customizable workflow patterns.
  • Create unique properties to classify tickets.
  • Report build status directly into the bug tracker.
  • Comprehensive notification system for custom alerts.

Cost: $70 for 3 Users, Free Trial Available

30. JIRA

@Atlassian

JIRA

JIRA is an instantly recognizable name in the software industry, and many of the leading web apps today explicitly integrate with JIRA workflows. JIRA is both a planning and tracking app, but at its core, it acts as a complete one-stop solution for companies to manage all of their project tasks, issues, and workflows, into a single environment. JIRA Software is instrumental in building and maintaining the best software on the market.

Key Features:

  • Native mobile apps to support ongoing progress.
  • Concise scrum boards for agile teams.
  • In-built kanban boards for tracking cycle lifetime.
  • Thousands of addons to increase functionality.
  • Custom query language (JQL) for filtering out content from the whole of database.
  • Integrate JIRA Software with all of the tools that your team is already using by leveraging its robust set of APIs.

Cost: $10 for Small Teams, and $75 for Growing Businesses

31. Lean Testing

@DamnBugs

Lean Testing

Lean Testing is a freemium management solution for tracking test process and provides a solution for concise bug and issue tracking. The hosted issue tracking is built to support productive teams of developers who have the need for a universal bug tracker solution. Lean Testing helps teams to create, maintain and execute test cases simply and efficiently.

Key Features:

  • Browser extensions to help report bugs on the same page.
  • Complex bug reporting details built with template support.
  • Create and manage user permission effortlessly.
  • Friendly user interfaces for easy accessibility and customization.
  • Data visualization using charts and reports.

Cost: Free with some limitations.

32. Bug Track

BugTrack

Bug-Track is a cloud-based software for tracking bugs and issues that revolve around the specific project, of which you can create unlimited using this platform. Built to support modern browsers and devices, this solution provides an easy fix towards tracking bugs and getting everyone on the team to collaborate towards fixing them.

Key Features:

  • Easy migration from other solutions with data import.
  • Compatible to track tests, bugs, and development requirements.
  • Take screenshots with a single click and paste them into the bug report.
  • Drag and drop user interface.
  • Flexible towards customization based on your needs.

Cost: Free Trial, Basic at $50

33. Sifter

@SifterApp

Sifter

Getting your codebase to a solid state can often be a hard task, especially when you’re only in the early stages of development. Sifter is a beautifully built bug tracking solution that helps software developers to understand how their code is progressing over time. With Sifter, businesses of all sizes can enjoy a real-time environment of issue tracking and bug submissions.

Key Features:

  • Dashboard that highlights critical issues first.
  • Status bars help you quickly gauge the progress for different batches of issues.
  • Easily create and bookmark lists of filtered lists of issues.
  • Automatically receive a daily summary of the previous day’s activity every morning.
  • The set-up takes a few minutes, and you can be up and running the same day.

Cost: Free 14-Day Trial, Small Plan at $29

34. Airbrake

@Airbrake

Airbrake

Airbrake is another one of those platforms that provide bug tracking as the secondary solution towards the main goal, which in this case is monitoring of crucial system and application errors that are streamlined in real-time as they appear. Airbrake automatically allows for engineers and developers to dive deep into the core issues and use the bug tracking system as a means of communication and error solving.

Key Features:

  • Consistent monitoring of code trends to retain quality.
  • Create custom workflows to promote productivity of your team.
  • Algorithm that recognizes similarities between errors and bugs.
  • Tap into the pool of code reports to find the cause of a particular exception.
  • Robust search and filtering let you find the exact errors you care about.

Cost: Startup at $99, Growing at $249

35. Gemini Tracker

Gemini Tracker

Gemini provides a great deal of solutions for software developers, most notably they are: issue and bug tracker, agile boards (scrum), support ticketing, and project management. Whether you are working in a Scrum/Agile environment or a more traditional one, Gemini Bug Tracking lets you control the releases of your fixes and enhancements so that you always know what is expected to go out in any given release, and who fixed what and when.

Key Features:

  • Versatile search filters that can be saved for future use.
  • Search for syntax across multiple projects instantly.
  • Get a visual overview of your most important metrics.
  • Instant messaging widget that accompanies your present workspace.

Cost: Free Trial

36. Trac

Trac

Edgewall is behind a ton of major software projects, and the Trac Project is not an exception when it comes to quality, reliability, and pitch-perfect functionality based on today’s modern standards. Nevertheless, Trac keeps things as simple as possible, with a major focus being dedicated towards the actual ticket management system so that bugs can get prioritized as soon as they appear.

Key Features:

  • Bug tracker and Wiki all within a single solution.
  • Project progress tracking through a history timeline.
  • Plugin functionality using custom components.
  • Custom logging of system messages using Python.
  • Ticketing system that works with bug reports feature submissions, task management, and support issues alike.

Cost: Open Source

37. Flyspray

Flyspray

Flyspray is a minimal and efficiency oriented bug tracking system built with PHP to be used in a web-based environment. While still in active development, Flyspray stands out with simple user interface that focuses around bug submissions and also tests reports that may have failed. Available for download from GitHub, and installation is an absolute breeze.

Key Features:

  • Support for MySQL and PostgreSQL.
  • Create and manage multiple projects.
  • Subscribe to individual tickets for notifications.
  • Easy to customize templating engine.
  • Roadmaps for achievements and version control.

Cost: Open Source

38. PR Tracker

PR Tracker

PR-Tracker is a Windows specific bug tracking software used by corporate and enterprise businesses across the globe. The configuration methodology is straightforward and allows companies to setup the software while gradually improving the configuration upon which the core of the system is going to run. It’s meant to be improved as per specific needs of your particular business.

Key Features:

  • Multi-project support with individual data entries and configs.
  • Create custom permissions for users based on their project assignments.
  • Powerful query support for sifting through your database.
  • Custom reports engine to create comprehensive analysis timelines.
  • Redundant data storage for speed and data corruption recovery using the database repair utility.

Cost: Starting at $199

39. Caravela

Caravela

Caravela primary focus is simplicity, in fact, the user interface is so simple, we had to check twice whether this was a functional app, and sure enough, it is. Caravela is a hosted solution, meaning that you can create your bug tracker within minutes using the in-built web-based setup process. The actual bug tracker is easy to follow and manage and provides a seamless experience for personal projects.

Key Features:

  • One-click installation for a new bug tracker.
  • Make your bug tracker public or private.
  • Create custom roles for each member of your team.
  • Smooth browsing experience without fluffy content reloads.
  • Visual charts show you how much is yet to be done.
  • Markdown-friendly ticket submission dashboard.

Cost: Free

40. NetResults Tracker

@NR_Tracker

NetResults Tracker

NetResults Tracker provides a wide range of solutions in-built with a single web-based collaboration software package. With thousands of existing customers, NetResults provides companies with an ability to manage projects, track issues and bugs, create custom knowledge bases, and enable a global support & helpdesk system that will enhance customer experience with the product you’re selling.

Key Features:

  • Multilingual support for all issues in real-time.
  • Integrated support for multiple applications.
  • Site Wide access to reports and links.
  • Scales based on the growth of your business.

Cost: Starting at $10

41. TestTrack

@seapine

TestTrack

TestTrack provides issue management tools for the most demanding projects in an infinitely capable and configurable platform. And with high scalability, TestTrack accommodates your needs seamlessly as your number of users or issues grows.

Key Features: 

  • Create, organize, and track hundreds of thousands of issues and work items.
  • Track results and measure progress.
  • Track every detail pertaining to every issue for greater visibility.
  • Tabs, folders, custom fields, and more for organization.

Cost: Free trial available; contact for pricing

42. BugHost

BugHost

BugHost has been providing bug tracking solutions for thousands of customers worldwide since 2001. With a pay-as-you-go model and no long-term contracts, BugHost is a flexible and affordable solution for developers.

Key Features: 

  • Track unlimited defects.
  • Maintains audit history for every defect.
  • Assign bugs to different team members.
  • Several pre-defined reports and search options.
  • Link similar bug reports.

Cost: Free 30-day trial; pricing starts at $29/month for five users and five projects

43. BugDigger

@bugdigger

BugDigger

Create helpful bug reports with the push of a button with BugDigger, which aims to help you save time and complete projects faster with smart, speedy bug reporting for web applications.

Key Features: 

  • Simple to create bug reports.
  • Automatic capture of screendump, web site usage history, and environment data.
  • Built-in editor to quickly markup screenshots.
  • Automatically send collected information to your issue tracker.

Cost: Plans start at $19/month for ten users and ten projects

44. Bugify

@Bugify

Bugify

Bugify is a self-hosted solution that easily integrates with your existing source control system for simplified issue tracking.

Key Features:

  • Email notifications and real-time desktop notifications.
  • @Mention other users for collaboration.
  • Add labels to issues and prioritize by high, medium, or low priority.
  • Reference other related issues.
  • History tracking.
  • Saved search filters.

Cost: One-time payment of $70

45. UserSnap

@usersnap

UserSnap

UserSnap is a central place to organize your web projects, collect feedback, and track bugs. Thanks to UserSnap, you can ditch the sticky notes and endless email chains for good and start gaining valuable user feedback in one location.

Key Features: 

  • Real-time collaboration.
  • Contextual information.
  • Ad-hoc feedback widget.
  • Integrates with Slack, Trello, Basecamp, and more.
  • Report bugs directly in your browser.
  • Point-and-click annotation tools.

Cost: 15-day free trial; pricing starts at $69/month for ten users and three projects

46. Snowy Evening

Snowy Evening

Snowy Evening offers issue tracking that’s both powerful enough for developers and simple enough for your clients. Seamless GitHub and jsFiddle integration provides greater efficiency to your workflow.

Key Features: 

  • Simple, all-project workflow.
  • Seamless navigation to quickly move between projects.
  • Quickly add summaries of issues with sensible defaults.
  • Add full descriptions, set versions, milestones, and priorities when needed.
  • Simple filtering options with saved filter views.

Cost: Free for one private project; pricing starts at $9/month for three private projects

47. BugAware

BugAware

A completely customizable, web-based software program that you can configure to your specific needs, BugAware can be configured to track just about anything that needs tracking, from bugs to issues, help desk inquiries, and more.

Key Features: 

  • Create and edit tasks.
  • Search options to sort, filter, and find bugs or tasks.
  • Generate reports and set email alerts.
  • Manage internal and external users.
  • Support queues.
  • System/project and task configuration options.

Cost: One-month free trial; pricing starts at $7 per user

48. informUp

@InformupForYou

informUp

informUp’s bug tracking system is a web-based software for managing unlimited projects suitable for small and medium-sized companies. With a one-click screenshot tool and online image editor, informUp makes it easy to track and collaborate on bugs.

Key Features:

  • Generate bugs directly from test runs.
  • Customizable with a powerful rule engine.
  • Link requirements, test cases, and defects.
  • Customized and drill-down dashboard.

Cost: Pricing starts at $7/month (hosted), Free for five users (self-installed)

49. Yodiz

@yodiz_team

Yodiz

An Agile management platform for getting things done better and faster, Yodiz includes a backlog, intuitive dashboard, sprints and releases, collaboration, and issue tracking in one.

Key Features:

  • Centralized repository for simple backlog management.
  • Epics to represent business requirements; break down into smaller features.
  • Visualize, schedule, and plan current and future sprints.
  • Assign and prioritize functionality for releases.
  • Visual issue board for managing issues across projects, releases, and sprints.
  • Batch modifications and one-click import.

Cost: Free for up to 3 users; pricing starts at $5 per user per month

50. Retrace

@Stackify

Retrace

Last but not least, we can’t talk about bug tracking tools without mentioning Retrace, Stackify’s product that enables developers to rapidly identify bugs and improve application performance on QA/test and production servers. Easy to install and use, and ready to scale to accommodate your needs, Retrace offers the flexibility and the power today’s developers need.

Key Features: 

  • SaaS-based APM.
  • Single pane for code performance and metrics.
  • Integrated errors and logs.
  • Works with most common app stacks.
  • Shows deeper contextual information to help you quickly address bugs.

Cost: Free 14-day trial; pricing starts at $10/month (pre-production) or $25/month (production)

 

Why Production Monitoring Can Come Too Late in the Development Process

Erik Dietrich Insights for Dev Managers, Live Queue Leave a Comment

I’ve spent a number of years, now, writing software.  At the risk of dating myself, I worked on software in the early 2000s.  Back then, you couldn’t take quite as much for granted.  For example, while organizations considered source control a good practice, forgoing it wouldn’t have constituted lunacy the way it does today.

As a result of the different in standards, my life shipping software looked different back then.  Only avant-garde organizations adopted agile methodologies, so software releases happened on the order of months or years.  We thus reasoned about the life of software in discrete phases.  But I’m not talking about the regimented phases of the so-called “waterfall” methodology.  Rather, I generalize it to these phases: build, prep, run.

During a build, you mainly solved the problem of cranking through the requirements as quickly as possible.  Next up, during prep, you took this gigantic sprawl of code that only worked on dev machines, and started to package it into some deployable product.  This might have meant early web servers or even CDs at the time.  And, finally, came run.  During run phase, you’d maintain vigilance, waiting for customer issues to come streaming in.

Bear in mind that we would, of course, work to minimize bugs and issues during all of these phases.  But at that time with most organizations, having issues during the “run phase” constituted a good problem to have.  After all, it meant you had reached the run phase.  A shocking amount of software never made it that far.

Production Monitoring and Software Maturity

We’ve come a long way.  As I alluded to earlier, you’d get some pretty incredulous looks these days for not using source control.  And you would likewise receive incredulous looks for a release cycle spanning years, divided into completely disjoint phases.  Relatively few shops view their applications’ production behavior as a hypothetical problem for a far-off date anymore.

We’ve arrived at this point via some gradual, hard-won victories over the years.  These have addressed the phases I mentioned and merged them together.  Organizations have increasingly tightened the feedback loop with the adoption of agile methodologies.  Alongside that, vastly improved build and deployment tooling has transformed “the build” from “that thing we do for weeks in the end” to “that thing that happens with every commit.”  And, of course, we’ve gotten much, much better at supporting software in production.

Back in the days of shrink-wrap software and shipping CDs, users reported problems via phone calls.  For a solution, they developed workarounds and waited for a patch CD in the mail.  These days, always-connected devices allow for patches with an arbitrary quickness.  And we have software that gets out in front of production issues, often finding them even before users do.

Specifically, we now have sophisticated production monitoring software.  In some cases, this means only watching for outages and supplying alerts.  But we also have advanced application performance monitoring (APM) capabilities.  As I said, we’ve come a long way.

The Remaining Blind Spot

But does a long way mean we’ve come all the way?  I would argue that it most certainly does not.  In the industry, we still have a prominent blind spot, even with our dramatically improved approach.

Many shops setup continuous integration and automated deployment to internal environments.  This practice neatly fuses the build and prep phases.  In these internal environments, they run automated regression tests, manual tests, and even smoke or load tests.  They diligently exercise the software.  But they omit to monitor it.  And, in doing so, they preserve the vestige of the phased approach.  Now, instead of the three-legged “build, prep, run,” you have the two legged “build-and-prep, run.”

By not monitoring the software in production like conditions, shops set themselves up for a category of issues likely only to be seen in production.  Often, they’ll view this sort of monitoring in their lower environments as superfluous, since their testing strategy should surface any problems.  They’ll save the money and effort of monitoring for the production deployment.  But this monitoring can come too late.

Remaining Reactive

APM tools do a lot more than give you a jump on outages or performance problems.  They also furnish a lot of valuable information.  They reveal performance bottlenecks, show you the source of underlying errors, and help you quickly get to the bottom of issues.

But if you find yourself doing all of this in production environments alone, you’ve missed an opportunity to see all of this before you ever ship.  For example, imagine a scenario where your application severely under-performs because of a correctable mistake, such as a massively inefficient query.  But it doesn’t underperform enough to run afoul of your load and smoke testing efforts.  And furthermore, it displays the correct behavior from a functional standpoint.  So, in spite of this mistake, the code makes it through all rounds of testing.

In production, you don’t trigger any alerts.  At least, not at first.  But as things scale up, your users begin to report poor experience, and your APM tool starts logging more and more red flags.  You notice this, get to the bottom of it and discover the underlying problem.  From there, you fix the issue and make everyone happy.

But why did it need to get that far?  You’re reacting to something that you could have caught much earlier.  This needlessly wastes a good bit of time, money, and customer goodwill.

Backed into a Design Corner

But it can get even worse than that.  Notice that in the hypothetical scenario I just outlined, a relatively easy fix presented itself.  Team members tracked down the issue, resolved it, and presumably issued a quick patch.  But what happens when it’s not quite that simple?

I’ve seen plenty of instances where an unnoticed mistake becomes intractable.  People build on such mistakes with the assumption of a good foundation.  Then they build some more on top of what they built on top of the assumption.  Before you know it, discovering this mistake might mean a choice between just working around it and significant rework.  I think anyone that has spent considerable time working with legacy code can relate.

You can thus pay a terrible price for not catching a mistake like this early — one that extends far beyond finding it in production instead of just before shipping.  You can pay the price for not catching the error before sealing it into the foundation of your codebase.

We write all sorts of automated tests for this reason, and practitioners of TDD cite this as one of the core benefits.  Catch mistakes as close to making them as possible.  By failing to deploy monitoring capabilities in lower environments, you run the same kind of risk that you run by skipping these types of automated tests.

Getting Rid of Phases Once and For All

When software delivery hums along, we don’t have any real, distinct phases the way that we used to.  Instead of segmenting, we get into tight feedback loops of “build-prep-run.”  Organizations like Facebook have this automated the whole way and wind up with a fused use of their tooling.

In general, I would recommend doing everything in your power to make lower environments look as much like production as possible.  You might not get all the way there.  Take Netflix, for example.  It unleashes its “chaos monkey” because it can’t possibly replicate the mammoth scale it requires in any environment but production.  But it uses this tool to do the best it can regarding reducing risk.

You probably don’t deal with their kinds of scale and testing limitations, but you can take their example nonetheless.  They fuse build-prep-run and leverage their available tools to address all of those concerns simultaneously.  You should do the same.  If you’re monitoring your software in production, start monitoring it in your other environments as well to catch mistakes before they have a chance to fester.

What are PowerShell Commands?

What are PowerShell Commands? Understanding PowerShell’s Cmdlets with Code Examples, Tutorials, and More

Angela Stringfellow Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Definition of PowerShell

PowerShell is the shell framework developed by Microsoft for administration tasks such as configuration management and automation of repetitive jobs. The term ‘PowerShell’ refers to both – the shell used to execute commands and the scripting language that goes along with the framework.

The scripting aspect of it is similar to Perl programming. The shell is comparable to bash in UNIX, with Microsoft even incorporating commands such as man, ls, and ps for convenience.

PowerShell 1.0 was released in November 2006 for Windows XP SP2, Windows Server 2003 SP1 and Windows Vista. While, initially, PowerShell had to be manually installed, the latest version 5.0 is available default with Windows 10. So, you can just go to Cortana and type ‘PowerShell’ or navigate from the Start menu. Read this to learn more about which Windows version uses which PowerShell version.

PowerShell also comes with an Integrated Scripting Environment (ISE). The ISE screen is split into two parts – the top one is used to write the script and the bottom for running commands manually. The ISE gives you a GUI experience, with smart syntax suggestions, coloring, tab completion and error handling.

If you are a Windows administrator who has to perform user management, DNS configurations and other tedious tasks frequently, PowerShell is the tool for you.

Benefits of PowerShell over Command Prompt

Well, PowerShell certainly has more power! Command prompt is an interface available to execute simple DOS commands; most users have not explored it beyond ping, ipconfig or in the programming world, ftp. However, PowerShell is much more than that. While there are many differences between the two, here are a few important ones:

  1. PowerShell uses cmdlets, not commands. Now, cmdlets are not just a different way of calling the same thing, but they expose complex system administration functionalities such as registry management and Windows Management Instrumentation (WMI) to the user. This makes them far more useful than the command prompt.
  1. PowerShell is object-oriented. The data output from cmdlets are objects (an example of how object-orientation makes PowerShell attractive) and not just text. This provides more flexibility to play around with complex data.
  1. PowerShell is developed using the .NET framework. This allows PowerShell scripts to use .NET interfaces and extend features that are not provided by default through cmdlets. The other way around is also possible – embedding PowerShell scripts in .NET code.

More About Cmdlets

Cmdlets are lightweight commands used in the PowerShell environment. Most of the cmdlets in PowerShell use the Verb-Noun format.

For example, Get-Command, Update-Help, Start-Service, etc.

To know more about cmdlets and understand what differentiates them from commands, you can read this link. One of the fundamental differences that we will discuss below is its object orientation – it reads input from a pipe and outputs objects (not text) to a pipe.

PowerShell Cmdlet Examples

Let’s go through an example to understand how this works.

On the command line, when you run the Get-Service cmdlet, you get a list of services on your machine.

PowerShell Commands

You can further filter these just to show the services that are running:

Get-Service | Where-Object {$_.Status –eq “Running”}

PowerShell Commands

Here, PowerShell processes every record output that GetService throws evaluate whether the ‘Status’ attribute is ‘Running’, and filters accordingly.

Note: $_ refers to the current record/object in the pipe.

One thing this example shows is the cmdlet’s record-oriented feature. The processing of one record at a time gives the flexibility to interpret data in a more intelligent way than it would be possible with text streams.

You can filter the previous output further, to just display the ‘Name’ of the running services.

Get-Service | Where-Object {$_.Status -eq “Running”} | Select-Object Name

PowerShell Commands

 

To learn more about any command, use

Get-Help –Name <Cmdlet name>

Or Get-Help –Name <Cmdlet name> -Online

The latter opens up detailed help with examples on the msdn site.

Here, you can learn some basic commands to get you initiated into PowerShell. If you are a visual person, go for this video. For a detailed list, this would help.

Quick Intro to Aliases

Aliases are alternate names for your cmdlets. These are usually handy for frequently used commands and when you don’t want to type the whole Verb-Noun format. For example, Get-Command has an alias ‘gcm’, Get-Service has ‘gsv’, Where-Object has a simple ‘?’.

Many UNIX commands are set up as aliases by default. For example, cat->Get-Content, man->help, ps->Get-Process, etc.

Run Get-Alias to see the complete list of aliases available in PowerShell.

Of course, you can create your own aliases as well.

Creating a PowerShell Script

To create a PowerShell script, all you have to do it open a file, write your code and then save it. PowerShell scripts have a .ps1 extension. Your script can then be run manually or automated to run as a job every day to perform administration tasks. Get started here for creating simple scripts with looping and conditions.

Tutorials and Resources on PowerShell Commands (Cmdlets)

Through PowerShell commands and scripts, there is so much benefit to be gained for an IT administrator. Here is a list of use cases where an administrator can leverage PowerShell commands. From gathering information about servers to managing folders, processes, services, memory, network, software installations, and registries, there are tons of features that PowerShell encapsulates. Not to forget about its seamless integration with .NET.

Once you get over the initial learning curve, you can start exploiting the functionalities of PowerShell. When you’re ready to explore further, check out the following tutorials and resources:

SQL Server Express

What is SQL Server Express? Definition, Benefits, and Limitations of SQL Server Express

Angela Stringfellow Developer Tips, Tricks & Resources, Live Queue Leave a Comment

If you use SQL in your work, you have several options available, ranging from enterprise options to the free SQL Server Express. Monitoring SQL queries are just one of the many things you can monitor with Prefix, so today we thought we’d take a closer look at how SQL Server Express stacks up to the more advanced, paid options, its limitations, and benefits.

Definition of SQL Server Express

SQL Server Express is a free version of Microsoft’s primary relational database management system (RDBMS) – the SQL Server. Essentially, the SQL Server is a database management system that can be used to store and access the information stored in many different databases. SQL Server comes with an impressive range of features like business intelligence, reporting, and in-depth advanced analytics.

The enterprise edition of SQL Server competes against enterprise-oriented systems like Oracle Database (DB) and MySQL. SQL Server Enterprise comes loaded with features and can be too expensive for smaller sized companies to maintain. Microsoft does, however, offer SQL Server in some different editions including:

  • SQL Server Enterprise
  • SQL Server Business Intelligence
  • SQL Server Standard
  • SQL Server Web
  • SQL Server Express

SQL Server Express is the most basic offering available. It is a full database engine you can deploy to a server or embed into an application. Express is free and comes with many of the same features as the enterprise edition. SQL Server Express is probably most suited to supporting production applications for smaller to midsize businesses. A typical SQL Server Express use case would be a deployment by developers who do not want to create applications with a database hosted on a server. Using Express, they would be able to develop apps with their SQL Server database.

SQL Server Express Benefits

Some benefits come with an SQL Server Express deployment.

  • Free: One huge advantage of SQL Server Express is that it is free. Your only outlay is the time investment you make downloading and setting up the system. If you only want to learn how to use SQL Server, then Express is for you. There is nothing to lose by downloading the system and getting used to how it works.
  • Scalability: SQL Server Express is an ideal starting point for smaller independent software vendors (ISVs) since it can be used with any smaller application. The licensing allows Express to be included as part of an app or product. While there are limitations around memory and socket usage, they are not as restrictive as some might think. Express is not limited to a single user which is a commonly held misconception. There is a 10GB database restriction, but that is a maximum size per database meaning you can have multiple databases that store up to 10GB of data. If you are an ISV and your company experiences high growth rate resulting in increased database demands, then you can only upgrade to a paid version of SQL Server.
  • Security: Within SQL Server Express there is the option of free online backup that will help to protect your valuable business data if anything goes wrong. Administrators should still follow security best practices like restricting access to backup folders and following Windows password policies.
  • Features: While Express is the “lite” version of SQL Server, there is still an impressive range of features that you would have to pay for with other systems. Express supports Full-Text Search, native XML, and the SQL Common Language Runtime. Other key features include a reporting component and report designer enabling custom report creation.

SQL Server Express Limitations:

Before deploying SQL Server Express, you should make yourself aware of its limitations which include:

  • 1GB maximum memory used by the SQL Server Database Engine
  • The maximum size of each relational database is 10GB
  • SQL Agent is not included in Express. The SQL Agent is a background tool which enables administrators to automate tasks like backing up data, database replication setup, job scheduling, user permissions, and database monitoring.
  • The limit on the buffer cache for each instance is 1MB of RAM.
  • The relational database engine is restricted to the lesser of 1 socket or 4 cores.

Versions of SQL Server Express

The 2016 version of SQL Server Express was released in three different editions:

SQL Server 2016 SP1 Express: This is the core express offering and comes with the features we have already laid out. In addition, capabilities like dynamic data masking and row-level security are now available in Express.

SQL Server 2016 Express with Advanced Services: This edition of Express contains the most features. Included are the relational database engine, Management Studio, reporting, and full-text search. Ideal for developers working on smaller applications which need to implement reporting on their applications.

SQL Server 2016 Express LocalDB: This edition is mainly used as a development tool. It is a lighter version of Express that runs as a local app. It does support all the main programmability features of the other editions. The big advantage of LocalDB for developers is that it allows them to create a local environment that is much easier to install and manage. Users can just start up an instance of LocalDB runtime as they need it and do not need to install a service.

SQL Server Express Tutorials and Resources

To learn more about SQL Server Express, check out the following tutorials and resources:

Metrics for Measuring Software Productivity & Efficiency

22 Software Developers and Development Leaders Reveal the Best Metrics for Measuring Software Development Efficiency and Productivity

Angela Stringfellow Insights for Dev Managers, Live Queue Leave a Comment

We’ve talked a lot about Agile development and DevOps — particularly, the always-pressing need to rapidly ship new versions of their code and update their products. But when it comes to evaluating the productivity of your development teams, what metrics matter most? Should you merely be concerned with deadlines and timesheets, or should you focus on customer satisfaction to evaluate team performance?

It’s a tough question to answer. There are lots of numbers you can track, but is there a single metric that accurately reflects software development productivity? To find out, we reached out to a panel of software developers and development leaders and asked them to weigh in on this question:

What are the best metrics to measure software development efficiency and productivity?

Meet Our Panel of Software Developers and Development Leaders:

To find out what metrics you should be tracking to monitor development productivity and efficiency, read on to find out what our experts had to say below.


Allison GrapponeAllison Grappone

@ascendle

Allison is a Product Owner and ScrumMaster at Ascendle. Allison’s career is focused on managing projects with grace and success, always with an eye on process improvement and quality assurance. She combines strategic vision and tactical execution to improve bottom lines, business systems, client satisfaction and team effectiveness.

“The most critical question to answer when measuring software development efficiency and productivity is…”

“Is the client ecstatic?” Below are the metrics and measurements that will get you to a “yes.”

Meeting times

Stick to standard time limits for Scrum meetings. If you find your team is extending the Standup meeting times on a regular basis, then the stories in the Sprint were not written or prepared sufficiently before the start of the Sprint. If your Sprint Planning meetings are taking longer than expected, the team needs to spend more time discussing stories during Backlog Grooming.

If all team members do not fully participate in all Scrum ceremonies, the length of a Scrum meeting is not a true indicator of the health of a project. Any concerns a team member has about meeting times or participation should be discussed in the Retrospective. During the next Sprint, the team can course correct.

Measuring meeting times: Provide a time-tracking tool that makes it easy to record time separately for different meeting types. At the end of a Sprint, review the meeting times for each meeting type. Address the good, bad and the ugly findings during Retrospective.

Time spent on a subtask

The best team requires little prodding from the ScrumMaster as the workday clicks by. An established team should understand the flow of a Sprint board. It’s like playing mini golf if you putt and only make it half way down the greenway you’re going to have to putt again until you make it to the hole. The ScrumMaster should not have to nudge a team member to make that next putt.

Team members should be clear in their role in advancing the subtasks in a story. A good rule of thumb is that any one subtask should never be estimated at more than 4 hours. That way the whole team isn’t waiting around until one person completes a subtask. The team should have standard rules for alerting the team that their subtask has been completed, so the team knows when to take on the next subtasks.

Measuring time spent on a subtask: Throughout the day the ScrumMaster can use a Burndown Chart to understand the progress of the Sprint. If things look “off,” they can head to the task board to see which subtasks may be holding up progress.

Completing new features

The client does not need to have a pulse on the day-to-day productivity of the team. The true indicator of efficiency and productivity will be if new features are being introduced at or before the time the client expects them.

A successful team will provide the client with new features to test, play with and discuss on a regularly scheduled basis. Keep in mind that a client is likely using new features to advance the product within the company or the marketplace. They should be able to trust at the end of every Sprint the team will demonstrate one or more features that will improve the client’s product pitch.

Measuring new feature completion: Maintaining a Version Report provides a clear picture of the progress of a team and the development of the product. The Product Owner can use the report to track the projected release date for a defined version. The Product Owner should work with the client to determine which new features are to be included in a version. The Version Report can then be used as a tool for discussion with the client at the end of every Sprint, to show progress and manage expectations.


Ross SmithRoss Smith

@PITSSAmerica

Ross Smith is the Chief Architect for PITSS America. As an architect, Ross ensures that projects are appropriately scoped, planned, and documented. His goal is to make sure that the team understands exactly what the customer needs and that the customer understands what they are going to receive from engagement with PITSS.

“The most effective measure for efficiency is going to come from…”

Agile development practices.

  • Burndowns, for instance, measure how many development tasks are completed over time. The time is usually measured in sprints, which are usually two weeks long.
  • Sprints are created with a set number of tasks, and the burndown shows whether tasks are completed to stay on the 2-week schedule.
  • Agile burndowns help show ROI and progress in smaller bursts instead of long-term projects.
  • Another efficiency measurement for applications in production is how frequently defects are raised and how long they remain unresolved.


Skot CarruthSkot Carruth

@gophilosophie

Skot Carruth is the co-founder and CEO of Philosophie Group, Inc.

“To quote the Agile Manifesto…”

“Working software is the primary measure of progress.” But today that isn’t sufficient—shipping software that works but doesn’t create value is not a good measure. The best metrics to measure the productivity of your software development are the metrics that you use to measure the business results. And the best measure of how efficient your software development is how quickly your software improves business results.


Jonathan RogerJonathan Roger

@AndPlus

Jonathan D. Roger is the Operations Director at AndPlus, LLC, a custom software consultancy just outside of Boston. With experience in government, finance, and green tech, he has a laser focus on customer delivery and process improvement. He is a Certified ScrumMaster and loves coaching Agile teams to reach their full potential.

“Software team productivity is an inherently difficult thing to put metrics — at least, quantitative metrics — around…”

Lines of code, bug rates, etc., are not necessarily good indicators of how well or poorly your software team is doing, especially if they are working on very complex problems. At our firm, we use a combination of qualitative and quantitative metrics to see that our teams are performing efficiently and being productive. To wit:

  • Customer satisfaction: The most important thing for us is that customers are happy with the work we are doing. Regular check-ins to ensure that the client feels that we are making adequate progress are crucial metrics for our team. The Scrum process that we use at AndPlus ensures that we demonstrate progress for clients every two weeks, and this gives us a perfect touch point with them.
  • Peer Code Reviews: Every line of code that gets put into a project at our firm goes through a peer code review, and our seniormost members of the technical team will also spot-check projects to ensure that code quality is being maintained. We will also compare the amount and quality of code written to the amount of time spent and logged on an issue — this gives us a feeling of how productive (or not) an engineer is being.
  • QA Kickback Rate: Once a ticket is dev-complete, we count on our engineers to ensure that the feature works. Once they are confident, they will push the issue to our QA team for review. Kickbacks from QA to the engineering team are common, but if we see a significant number of issues (especially simple issues) being kicked back more than once, that is a leading indicator of problems with the engineering team’s effectiveness and productivity.
  • Time Logs versus Historical Data: After several years of writing custom software, we have thousands of completed issues in our JIRA instance, all associated with time logs. We can use this data to track historical time records for story point levels (e.g., the median 2-point user story takes n hours from birth to death). While any one user story may go far longer or take far less time than the median if we see a large number of stories taking longer than the average it is an indicator that the team may not be as performant as they should be. On the other hand, if a team is consistently taking less time than the median, it indicates either a highly-performant team or a team that is padding estimates.

At the end of the day, our goal is to be fair to our engineering team and our clients — we know that every project and every issue within a project are different, and complexities arise even when the team is working on something that should be simple. As a custom software firm, everything we do is in some way novel, and we take that into account as well.


Vlad GivertsVlad Giverts

@claralending

Vlad Giverts is the Head of Engineering for Clara Lending, a technology-enabled lending platform making financing a home easier, faster and more affordable. Before joining Clara, he was a partner and Sr. Director of Software Engineering at Workday Ventures. He holds a B.S. in Computer Science from the University of California, Berkeley.

“The truth is, there’s no good way to measure software development efficiency and productivity. But you can measure things that have a positive or negative effect on productivity….”

It’s very much dependent on the type of software and the structure of your team and organization.

That said, there are some common measures that can be useful to consider to see if they apply to your situation:

1. Bugs. Not how many, as there will always be some bugs, but rather how much time you’re spending on them each week. This includes both fixing issues once you’ve identified them or troubleshooting issues when they come up. If it’s more than 20% of your engineering time, you might have a quality/architecture problem that is a drain on your productivity.

2. Uptime. Related to the above. If you have a product on the internet, how much of the time is it unavailable to customers? Every time that happens it’s a distraction to the engineering team (and a cost to your business!).

3. Time from code complete to done. This is a tough one to measure, but incredibly impactful if you can improve it. When a developer is done writing the code for whatever new feature, there’s always some steps before those features are available to your customers. Those steps could be a code review, running the build including automated tests, QA and/or User Acceptance Testing, and the actual deployment/release process. Any one of those steps could result in an issue that requires the developer to go back into the code to fix or change something. Depending on how long this process takes and how many times the developer has to get back into the code, it could be a massive drain on productivity as the developer could have moved on to something else and has to context switch back and forth and rebuild the context every time.


Michael MahMichael Mah

@MichaelCMah

Michael Mah teaches, writes, and consults with technology companies on estimating and managing software projects, whether in-house, offshore, waterfall, or agile. He is the managing partner at QSM Associates Inc. and director of the Benchmarking Practice at the Cutter Consortium, a US-based IT think-tank. With over 25 years of experience, Michael and his partners at QSM have derived productivity and quality patterns for thousands of projects collected worldwide. His work examines time-pressure dynamics of teams and its role in project success and failure. His degree from Tufts University is in physics and electrical engineering, and he is a mediator specializing in conflict resolution for technology projects, having completed his certification from the Program on Negotiation at Harvard Law School. Michael is also a private pilot and lives in the mountains of Western Massachusetts. His non-profit work is with Sea Shepherd Conservation Society.

“The global software industry is estimated at over $US 400 billion while adding an estimated US$ 525 billion to US GDP alone…”

Naturally, some leaders ask themselves, “What’s the best way to measure software development productivity and efficiency?” The truth is, there are no best metrics. That said, there are some that are more valuable than others.

The question is, what would you do with these measures in a perfect world? Would you want to diagnose and understand how to improve? Would you want to compare different teams? Would you want to more reliably estimate future projects and make management trade-offs?

Some might reply, “All of the above.” But another key factor could be missing. What is the value of the software that teams produce (an even trickier question)? Are you looking to cut costs, drive top-line revenue, and/or optimize your capabilities to grow the company and its staff?

Whatever your answer might be, you’ll have to understand that there are several dimensions which matter. Here are a few things to consider:

  • Productivity is often seen as delivering functionality at a lower cost.
  • It is also seen as delivering the same or more functionality, faster.
  • Sometimes these are inversely related. For example, delivering at a lower cost, but taking more time.
  • Alternatively, one might achieve a faster schedule, but they accomplish this with large teams, at a higher cost.
  • Lastly, some might say that for a given schedule and fixed cost, a team, delivers MORE than it did before.
  • Hence any measure MUST take into account all of these factors. 1) Speed 2) Effort/Cost, and 3) Volume of Functionality Delivered.

Some measures since the beginning of measurement looked at only two out of three, using metrics like cost per unit function, or velocity measures like story points per 2-week sprint. Each of these measures leaves out another dimension, and that’s where things get misleading. So now it becomes clear that ALL THREE have to be taken into account, while not omitting either effort or schedule. Make sense?

But wait. There’s more. None of this can hold water if teams deliver software at higher productivity, but at the poorer quality (more defects or poor usability) or low value.

In other words, higher productivity means nothing, if you’re delivering more junk that no one cares about, which has little value. That said, if you know what you’re doing, higher productivity can mean a whole lot. But that’s the subject of another article.


Swapnil BhagwatSwapnil Bhagwat

@swapnildigital

Swapnil Bhagwat is Senior Manager – Design & Digital Media and implementing web, design and marketing strategies for the group companies. He is an MBA graduate with work experience in the US, UK, and Europe. Swapnil has worked for more than a decade across a range of businesses for the global markets.

“Some of the Agile methodology metrics that are now being regularly used to measure software development efficiency and productivity include…”

Cycle time: The time taken to drive a change in the application and deliver it into production.

Lead Time: The duration between the formation of an idea and its delivery.

Fix rate: The time taken to open and close a specific production issue.


Rodolfo JustoniRodolfo Justoni

@NearshoreTalent

Rodolfo Justoni is a Project Manager (CSM) at Nearshore Systems, overseeing and defining Agile processes and team metrics with 15+ years background in the IT world in both technical and management roles. Rodolfo has experience managing agile teams and commercial engagements from the early stages and in leading teams within different locations and countries.

“The metrics we, at Nearshore Systems, use to measure development efficiency and productivity are the following…”

We measure efficiency as the percentage of an engineer’s contributed code that’s productive. The higher the efficiency rate, the longer that code is providing business value. A high churn rate reduces it. Code Churn is the percentage of a developer’s code representing an edit to their recent work. It’s typically measured as lines of code (LOC) that were modified, added and deleted over a short period such as a few weeks.

The primary purpose of measuring churn is to allow software managers and other project stakeholders to control the software development process, especially its quality.

The most prolific engineers contribute lots of small commits, with a modest churn rate, resulting in a high-efficiency rate. Understanding an engineer’s typical efficiency rate can help you understand their character and where they will fit in best.

As for the productivity, we take into consideration Epic and release (or version) burndown charts that track the progress of development over a larger body of work than the sprint burndown and guide development for both scrum and kanban teams.

Finally, we also measure velocity. Velocity is the average amount of work a scrum team completes during a sprint, measured in either story points or hours, and is very useful for forecasting.


Doru ParaschivDoru Paraschiv

Doru Paraschiv, co-founder and VP of Engineering at IRON sheep TECH, is an engineer with more than 15 years of experience. He has recently collaborated with DZone on research about optimization and monitoring tools.

“If we’re talking about team productivity (and not individual productivity), the best metric is represented by some on spec bug-free features delivered…”

By this, I mean that the way I know a team is doing the job is if:

  • The team delivers the feature.
  • The feature follows the specifications.
  • The feature is bug-free.

However, the subject is touchy, as there are a plethora of takes on this issue. In reality, it is truly hard to measure software development productivity. It all depends on the type of business you are running. For certain businesses, delivery speed would be the most important. For others, following the specifications would be of greater importance. And for others, the absence of bugs is the best one. In the end, it all depends on the type of software you write.

For instance, for us, productivity doesn’t necessarily mean speed of development, but rather a robustness of delivered code/features. As an example, a team would write a feature, but then that feature would evolve in time, grow, become more complex. Productivity would be writing that feature, so we do not find ourselves in the future needing to re-write the feature because we didn’t make it flexible.

So in our case, the following are most important: the flexibility of code and the extensibility of code which allows us to incrementally upgrade a feature without totally re-writing it.


Cristian RennellaCristian Rennella

Cristian Rennella is the Co-Founder of elMejorTrato.com. They have eight years of experience in online entrepreneurship in South América, where they have developed the biggest education company, now with 134 employees and with more than 21.500.000 unique visitors working internationally in Brazil, Argentina, Chile, México, and Colombia.

“The best ways to measure software development productivity and efficiency are…”

1) For me, being busy means doing stuff, being productive means getting stuff done. Every person in my company knows that everything is measured with things that are finished (it does not matter if it’s a huge product release or just adding a button).

2) We do pair programming, so a great way to measure software development is when each person agreed that there is nothing more to improve. This makes the quality of the code much better.

In the end, productivity (point 1) + quality (point 2) is the key to our success.


Tosho TrajanovTosho Trajanov

@TsTrv

Tosho Trajanov is the co-founder of a tech startup, ADEVA, aiming to help startups and businesses build and manage their software development teams. Tosho is a software engineer with extensive experience in building teams and developing complex cloud solutions & enterprise web applications.

“In order to offer better service to our clients, in the last few years I have been researching about software development productivity and efficiency. My findings are simple…”

There is no formal and objective measurement of efficiency and productivity of software development any
organization could use straight away.

Instead of using KPIs by the book, at Adeva, we started scheduling short meetings with the developers we are evaluating, listening closely what problems they are facing with. We are also asking them how helpful and knowledgeable the rest of the team members are to identify the team performance in general. We also encourage them to think about their productivity and efficiency and come up with ideas that could help themselves as well as the other team members.

We also use some tools to analyze how a developer code integrates with the codebase and make a quantitative determination of the developer’s efficiency. We also review the code’s performance, security and whether the code is going to have a lower cost of maintenance in the long term.


Andrew WardAndrew Ward

@scorchsoft

Andrew Ward is the 2016 winner of the ‘Most Influential Male’ award at the Silicon Canal Tech Awards, and Birmingham Chamber of Commerce’s ‘Future Face of Entrepreneurship.’ Andrew is the Managing Director of an app development agency called Scorchsoft, and the CTO of MODL, a disruptive platform for booking professional models.

“You need to be extremely careful when measuring efficiency and productivity within a tech business, and just because you are agile, doesn’t mean you are effective…”

To give you some context, I run two tech businesses in the UK, and though measuring effectiveness is critical for both, the way I approach this for each business is very different.

One company develops web and mobile apps for its customers. It’s service based, so delivering projects on-time and to a budget are the most important metrics. As most clients expect us to agree to deliver a product at a fixed cost, this means that we usually have to define a clear, unambiguous specification right at the start. Productivity is measured regarding development hours, and we use tools such as Toggl and Jira to meticulously track this on a daily basis.

A project is a success if the customer is satisfied, and we deliver on or under the quoted number of development hours. If a project does go over, then we measure this regarding the opportunity cost of not being able to spend that time on other paying projects – which works out to be very expensive. Though other metrics exist, such as lines of code used, code quality, the number of tests passed, or the number of external libraries imported, none matter if the project goes over-budget, as even small over-runs can consume the entire profit margin.

The second business is product based, meaning the success of the company is not tied directly to the time features take to develop. If a feature takes twice as long then we may have to delay the launch of the next release, but the cost to the business is the based cost of someone’s salary, unlike the development business, there is no opportunity cost to compare this against.

With this business, we can work agile. We still record time and get developers to estimate hours to hold the team accountable, but you have to be careful treating developers like machines. A shift in frame of mind can be the difference between fixing a bug in five minutes, and being stuck on it for days. In this environment you stand to lose a lot more by management, myself or the other directors, making a poor decision. For example, if we develop a feature that the client does not want, then we could waste months of development time with one bad decision. This has a much larger impact on the business when compared to a developer taking slightly longer.

In this environment we follow the build, measure, learn approach proposed by Eric Ries in his book The Lean Startup. We focus more on marketing metrics such as conversion rate, user behaviors, or customer feedback scores. In working agile, there is more autonomy on the team to make decisions throughout the development process, and strategic decisions are made and tested on a daily basis.


Dawn RobertsDawn Roberts

@DawnTheMaven

Dawn Roberts is a business owner/entrepreneur of a consulting company, Dawn Roberts Consulting. Her niche is business and personal efficiency — specifically process streamlining, complex problem solving, efficient mindset development, and value leakage/waste removal. She’s saved well over $6 million in efficiency improvement projects she’s worked on thus far. She has also developed a 4-week online class targeted at empowering individuals to be more productive and effective, delivering a higher volume of value-based activities.

“I recommend ranking business softwares based on four criteria…” 

  • Opportunity – This will include how much time and cost it will save for your business from automation. Not all processes will fit well with program automation. The opportunity should be quantified in clear terms up front.
  • Potential Challenges – This will include any downsides of automating, or of the software application itself. Often, there may be incompatibilities that should be flagged up front.
  • Cost to Implement – This will include the cost of the program and any associated IT costs in implementing. I encourage people to have an exhaustive list here. I’ve seen budgets blown because unexpected costs creep up during implementation that wasn’t planned for. Sometimes these costs become so high that the project would have been reconsidered up front if the costs had been known.
  • Time to Implement – This includes how much time it will take to fully integrate the program. Include staff time and support time from the automation company. Be realistic here if you work for a large company, as integrating IT software can take quite a while in some cases, and can impact project ROI.

These will all differ business to business and application to application. You can create a weighted ranking for these four criteria, or you can create a numbering system for each that will allow you to sort and filter the different options you are looking at.

When looking for software that is aimed to increase automation or efficiency, I recommend the following features:

Easy and intuitive to use:

  • No programming required
  • Drag and drop user interface
  • Online training videos available for staff

Scalable and transferable:

  • Remote deployment capabilities
  • Ability to be scaled based on future process needs
  • Ability to be easily modified based on future process needs
  • Ability to be used in different business applications once introduced

Smart:

  • Reporting capabilities – Easily see process health metrics that you assign.
  • Analytical capabilities – Easily analyze different parts of the process.
  • Optimization capabilities – Program should automatically flag optimization areas.
  • Technical support – Technical support should be readily available for the program.


Gady PitaruGady Pitaru

@BadgerMaps

Gady Pitaru is the Chief Technology Officer at Badger Maps. Badger is a sales routing software that helps field salespeople be more successful.

“Measuring software development efficiency and productivity depends on the type of organization…”

Consulting firms will tend to measure efficiency and productivity per project more quantitatively since every hour will be billable. Software product companies might not be able to measure efficiency and productivity as easily, so different project management methodologies can help. We use the Scrum methodology at Badger, which includes a built-in way of measuring software development efficiency and productivity at the team level using a team’s velocity (Scrum stresses team collaboration). Velocity is basically how much work a team can do in a period, and over time can become a good average for measuring how efficient or productive a team is.

With that said, any single metric will never perfectly reflect reality, so at best they are an estimate instead of a perfect measure. A better measure of software development efficiency and productivity is simply to look at how well the business goals are being met. Instead of counting hours or trying to squeeze every last drop from a single hour, you can instead look at how the software development efforts contribute to meeting the overall business goals. That measures the efficiency of a whole organization instead of just a single work group.


Steve KrzysiakSteve Krzysiak

Steve Krzysiak is a developer/manager with over 16 years of experience. He’s had the opportunity to manage many remote employees, including offshore 3rd shift developers who require extra attention. He has always looked for ways to quantify developer output and has discovered some approaches that work well. Steve is the founder of On The Road Creative, an agency that travels to clients for discovery phases then finishes the work remotely.

“First and foremost, it is important to communicate to the development team that metrics are not a form of micromanagement, but a professional self quantification that will help devs grow…”

I tell teams that there are no penalties, only introspection and voluntary behavior changes when unconstructive patterns emerge. If you’ve built a solid team then they’ll be on board with this; otherwise, they’ll see it as micromanagement and just game the system. The prerequisite is to build a passionate team of individuals that are inclined to individual growth.

Regarding metrics, there isn’t one metric to rule them all. There are many that I have looked at over the years to spot time wasters and anomalies. Time wasters are often areas where work can be optimized, while anomalies provide insight into a developer’s potential distractions (e.g., personal issues, professional mindset changes). I do not use nor encourage software metrics to be used to assess the quality of a developer.

Moving on the metrics themselves, here is a list of ones I have found useful. I have also included general productivity tips as well.

1. Lines of code (LOC) – Although this is often the most misleading metric it can also provide a baseline for the right project. Some projects are more code-intensive; some are more debugging intensive. A manager will see in a few weeks whether or not LOC is a suitable metric for their project. If the whole team is all over the board with these numbers then likely this metric is not suitable for the project.

2. Commits/Check-ins – This also can be similarly misleading, but for the right project, unless there is a predictable cadence established early on then it can be a strong assessment of work done.

3. Time tracking – The buy-in from your team on this one can be hard. It has to pitch in a manner that this is used to grow individually and identify bottlenecks in the software, as opposed to identifying slow developers. With a good team, this buy-in comes easy. This works only if the manager trusts his team.

4. Pull Requests (i.e. code reviews) – Although not all shops have a pull request (i.e. code review) model in place, for the ones that do, it can provide insight into how to prevent code issues in the first place. After awhile you will notice repetitive PR suggestions/comments that can be avoided in the future even before it wastes another developer time.

Imagine a case where there are four suggestions to do something one way as opposed to another. If the suggestion is something that can be summed up in a code linting rule, then the developer will see their error before they make a future PR and not waste others’ time. Also, if people are nit-picky with their PR suggestions, it’s often a good indicator that there might be some ego issues on your team.

5. Version Control System (VC) History – Provides a sense of past productivity. If a codebase is old enough, you can use a tool like CodeScene.io. CodeScene will look at the VC history and identify ‘hotspots’ in the code, among other things. Places where logic changes happen often. This can allow the team to break up the code and/or be mindful of the sensitive nature of those files. I encourage you to look into the CodeScene product some more, they are the most promising tool I have come across, and one of the founders gives a good talk about what forensic psychology applied to codebases, which is what code scene is built on.


Rob ZuberRob Zuber

@z00b

Rob Zuber serves as chief technology officer at CircleCI. Before CircleCI he was a co-founder of Utah Street Labs and was the chief mobile strategist of Critical Path. He works out of CircleCI’s headquarters in downtown San Francisco.

“The best metrics for measuring software development productivity and efficiency are…”

Commit-to-Deploy Time (CDT): This is the time it takes for the code to go from committing to deploy. In between, it could go through testing, QA, and staging, depending on your organization. The goal of measuring CDT is to tell CEOs (and engineering managers) how long it’s taking the code to get from one end of the pipeline to the other and what roadblocks you’re encountering if any.

Ideally, if you’re doing CI best practices, tests are good quality, have been automated, and you can get from commit to deploy-ready status in mere minutes, even seconds for a microservice. If you have a largely manual QA process, that will likely mean your commit-to-deploy time is longer and can reveal where you have room to improve.

These improvements could be more of the technical side (e.g. our tests are flaky) or more process-oriented (we use complex integration tests where only unit tests are needed) or some combination of the two. At any rate, your goal should be to improve your commit-to-deploy time in some increment.

Most fast-moving organizations (e.g., Facebook, Amazon) deploy hundreds of times a day. For smaller organizations, daily deployments would be a good goal. The smaller your commits are, the faster they can get into production, and the faster you’ll be able to fix things when they go wrong… and at some point, they definitely will go wrong. More frequent deployments will also get your team accustomed to doing so, which will hopefully mean they’ll get better and faster at doing it.


David AttardDavid Attard

@DARTCreations

David Attard is an established web designer and author on influential web design and development sites. He also manages the development of new software projects for various companies.

“I’ve found that the best way to measure development efficiency and productivity is…”

Team velocity as defined by Agile development.

There are various reasons why this works well.

1 – The unit of work is defined as necessary for the specific team. Whether this is an hour, a day of work, or the completion of a task, it’s the team who decides what best applies to them.

2 – The velocity takes into consideration such stuff as vacations, sick leave, support and other things which eat away from development time.

3 – Averaging the velocity over the last three sprints makes it easier for better prediction and measuring of efficiency (or lack thereof). By monitoring whether the velocity is going up, down or remaining stable, one can understand better how the development team is performing.


Kean GrahamKean Graham

@monetizemore

Kean Graham is the CEO of MonetizeMore, a leading ad tech firm that is a Google Certified Partner.

“The best way to measure software development productivity and efficiency is…”

We found that the point system on Jira Agile Scrum is the best way to gauge the performance of individual devs and as a team. We gauge the growth of the productivity of the team and can assess deadlines using this.


Dan RasbandDan Rasband

@danrasband

Dan is the Development Team Lead for Objective in Salt Lake City, Utah. He holds a master’s degree in Linguistics from the University of Hawai’i and received bachelor’s degrees from Brigham Young University in Korean and Linguistics.

“The best metric to measure software development efficiency and productivity is…”

How often you ship new code and features. Ultimately if you’re not shipping often, you’re not efficient or productive.


Hristo StalevHristo Stalev

@KanbanizeInc

Hristo Stalev is the CTO and co-founder of Kanbanize. Before starting the company with his partners, Hristo was specialized in front-end development.

“As a Lean organization, we try to continuously improve our process efficiency. The most important metrics for us are…”

The weekly throughput of the dev team, average cycle time of each assignment type, and waste time. The throughput consists of the number of cards that the team places in the done section of their Kanban board. We place special attention on how long each assignment takes to be completed. Out of the cycle time, we calculate how much time was accumulated on waste activities and try to minimize it.


Andrew HallerAndrew Haller

@ahaller07

Andrew Haller is a co-founder and co-CEO of AirDev, a San Francisco based startup that designs and develops custom software for websites and mobile apps. Haller, a native of Chicago, graduated from Stanford and Harvard Business School and lives in San Francisco.

“A good rule of thumb for software development efficiency is…”

How long work actually takes relative to original estimates. Sadly the industry has a reputation for running over budget and past deadlines, so while all development shops can set ambitious targets, those that can deliver (or provide fixed pricing and money-back guarantees) are typically the most efficient.


Steve MezakSteve Mezak

@accelerance

Steve Mezak is the founder and CEO of Accelerance, which connects companies that need software development services with the most qualified outsourcing firms around the globe. Mezak is a software development expert and co-author of  Outsource or Else! How a VP of Software Saved His Company and author of Software Without Borders.

“There are multiple ways to measure software development efficiency and productivity, depending on the goals of the organization…”

To emphasize or measure software development efficiency and productivity, several of our clients focus on:

1) Hitting Release Dates – The team’s ability to agree to a product roadmap and then hit the dates for releases. There is some give and take when using an agile development methodology. In some cases, the release date is the most important target because of related marketing, promotional and publicity campaigns. In other cases, the implementation of a specific feature is required for a successful release, whether it is on the target release date or later. The most important thing in this context is the ability of the development team to communicate with product management which is best measured by hitting release dates.

2) Quality and Customer Satisfaction – Happy customers who are now promoters, willing to recommend the software to others because it delivers value and has no serious deficiencies or bugs.

3) Profitability – Increasing revenue and/or cutting costs. This could be the increasing value of the software over time to support increasing the revenue per unit, such as the monthly subscription price for end users. You can also reduce expenses by increasing productivity of the team and possibly by outsourcing.

Stackify Rising Star and User Experience Awards 2017

Stackify Wins 2 Awards in 2017

Natalie Sowards Live Queue Leave a Comment

Stackify’s Retrace APM gets distinguished with the Great User Experience Award and Rising Star of IT Management Software Award for the year 2017.

With a moderately long, but notably positive record of helping developers track logs and fix bugs, Retrace is often referred to as the ‘pleasant surprise’ in the IT management software industry. There is no sentence we’d prefer to hear at Stackify, and no greater motivation to keep the good work up.

Retrace was featured on one of the best-known online B2B software directories FinancesOnline.com. The platform’s team recognized our Retrace’s potential and submitted it to a standardized scoring procedure to check whether it meets the criteria to be recommended to their large base of readers.

Results couldn’t be better: FinancesOnline awarded Retrace with two of their most prominent 2017 Awards: The Great User Experience Award, and the 2017 Rising Star online. Both valuable awards are reserved for products that facilitate IT management and give users an intuitive environment to work in. More importantly, they are delivered to vendors who make sure clients will have all necessary features to automate their day-to-day operations.
Receiving these awards was an honor for a team as young as ours, but what made us even happier was to get included on the platform’s list of SaaS software rising stars.

According to experts, Retrace provides developers with the flexibility they need in order to be more efficient and faster when dealing with logs and errors among other things. It is easy to set up, and it leverages a contextual approach to their metrics, errors, and logs, further simplifying the resolution process to application problems. On top of all that, Retrace monitors all applications and servers and tails logs in real-time.

Best VS Code Extensions

Top Visual Studio Code Extensions: 50 Powerful Tools to Make Visual Studio Even More Useful

Angela Stringfellow Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Microsoft’s Visual Studio Code provides developers with a robust marketplace packed with useful tools and extensions that make VS Code even more functional and customizable to meet every developer’s needs. New extensions are added regularly, and you can find tools, libraries of snippets, and just about anything else you could want to streamline the development process. In fact, Stackify Prefix has its own code extension in the Marketplace — the easiest and most powerful .NET profiler you’ll find.

Extensions are a vital part of the Visual Studio Code experience.  We decided to take a little tour of the other offerings in the Marketplace to come up with a list of 50 most helpful, must-try extensions (in addition to Stackify Prefix, which we, of course, consider a must-have) for any Visual Studio Code enthusiast. All of the following 50 extensions are free, although some require you to have other programs or services installed. We hope you’ll find some great ideas in this list of tools.

  1. TSLint

TSLint

If you’re doubtful about your TypeScript quality, verify it with this Visual Studio code extension, which integrates the tslint linter for TypeScript language into VS Code.

Key Features:

  • Checks the quality of your static TypeScript code
  • Analyzes the whole project or a particular section of your code
  • Allows you to auto fix errors when scanning lines of code
  • Supports automated fixing of warnings
  • Requires tslint (installed globally or locally)
  1. Auto Import

Auto Import

A Visual Studio code extension that automatically finds parses and provides you with code actions and code completions for every import you have, Auto Import works with TypeScript and TSX.

Key Features:

  • Easily scans code in large projects
  • Import status bar show you the number of imports available
  • Merge imports from the same location
  1. Docker Support

Docker Support

Docker is one of the hottest trends in development today, and if you use Visual Studio Code, you don’t want to miss out. This is the official Docker extension to help you create dockerfiles and add syntax highlights, commands, and snippets to them.

Key Features:

  • Makes creating and launching containerized applications easy
  • Automatically generates dockerfile and docker-compose.yml
  • Dockerfile and docker-compose commands are built into the command palette
  • Syntax highlighting and hover tips
  • Snippets and linting for dockerfile files
  1. Visual Studio Keymap

Visual Studio Keymap

Keyboard shortcuts don’t just save time but enable you to maintain your flow as you code. They’re very popular with Visual Studio users, and this extension makes it possible to use your favorite VS keyboard shortcuts in VS Code.

Key Features:

  • Provides the traditional Visual Studio keyboard shortcuts
  • Easy to add your custom shortcuts
  • Brings shortcuts of Windows, Linux, and Mac to Visual Studio code
  • Find all keyboard shortcuts in the extension’s contribution list
  1. EditorConfig for VS Code

EditorConfig

Maintain a consistent coding style with this Visual Studio code extension. Even if your teams use multiple IDEs and editors, consistency is achievable with EditorConfig for VS Code. Internally, EditorConfig uses the editorconfig npm package, one of the few EditorConfig cores available. Check out the main EditorConfig site here.

Key Features:

  • Official release by EditorConfig for Visual Studio
  • Define styles for indenting, tabs, and where lines begin and end
  • Text editor plugins for editors to read and follow defined style rules
  • Uses editorconfig npm package
  1. Evermonkey

Evermonkey

Evernote is one of the most popular note taking apps. It can be used to improve the quality and efficiency of your coding with the Evermonkey extension. This handy Visual Studio Code extension enables Markdown support for Evernote, following the same principles as Sublime Text but with a faster experience.

Key Features:

  • Provides support for Evernote in Visual Studio Code
  • It caches data locally to avoid repetitive net requests for a faster experience
  • UI design updates to be added to improve viewing
  1. React Redux ES6 Snippets

React Redux

Get React and Redux snippets using ES6 for JavaScript to help you structure components with this simple, yet helpful VS Code extension.

Key Features:

  • Provides snippets using JavaScript
  • Quick access to snippets by pressing Ctrl+Space
  • Assists with boilerplate as well
  1. Debugger for Chrome

Debugger for Chrome

Chrome maintains the most market share among browsers, and it’s widely used by developers thanks to its robust marketplace of tools and tricks for simplifying developers’ lives. This VS Code extension assists you in debugging your JavaScript in Chrome, offering several helpful, time-saving features.

Key Features:

  • Works with other targets that support Chrome debugging protocol
  • Sets breakpoints even on source files when source maps are enabled
  • Debugs script tags and eval scripts as they add added
  • Locals pane, watches, and console
  1. JSHint

JSHint

JSHint is a great tool to quickly spot errors with your JavaScript code. If you’re used to JSHint, you’ll also want to use it in Visual Studio Code, and this extension makes it simple and easy. You can check out the JSHint website here.

Key Features:

  • Uses the default JSHint configuration as described on their site
  • All files are included in this extension by default
  • Allows you to exclude specific files and folders from the review
  • Option to disable JSHint for specified workspaces
  1. C#

C#

Write great C# code for .NET Core with this extension, powered by OmniSharp. It even debugs your code and supports a variety of operating systems. VS Code aims to be a useful tool for cross-platform C# development, and in fact, many Unity game developers use VS Code rather than the MonoDevelop IDE, making this C# extension a particularly useful tool for many developers.

Key Features:

  • Supports debugging and lightweight development tools for .NET Core
  • Editing support for C# including syntax highlighting, IntelliSense, Go to Definition, Find All References, and more
  • C# debugger available on Windows, Mac, and many Linux operating systems
  1. TODO Parser

TODO Parser

If you have trouble parsing TODOs in the working files of your project, this Visual Studio code extension can help. It lets you parse multi-line TODOs and makes your workflow a lot more actionable.

Key Features:

  • TODO counter is available in the status bar of your current document
  • The counter provides list of all available TODOs
  • Option to parse single and multiple TODOs at once
  • Supports any programming language
  1. PowerShell

PowerShell

PowerShell is a great object-oriented programming language. If you use this programming language, this is a must-use extension offering PowerShell language support for Visual Studio Code.

Key Features:

  • Write and debug PowerShell scripts in VS Code’s IDE-like interface
  • Works on platforms like Windows, Linux, and iOS
  • Provides tools to find variables and cmdlets references
  1. Angular 4 and TypeScript/HTML VS Code Snippets

Angular 4 and TypeScript Snippets

Use this Visual Studio code extension to add code snippets of Angular (v2 or higher), TypeScript or HTML to your editor. It’s an incredibly easy-to-use extension that streamlines coding productivity.

Key Features:

  • Adds Angular version 2 and up, TypeScript or HTML code snippets
  • Multiple snippets provided in the description for both TypeScript and HTML
  • Instantly improve productivity in Angular 2 and up with these snippets
  1. Sublime Text Keymap

Sublime Text Keymap

Sublime Text is an excellent text editor for coding. If you are used to Sublime Text keyboard shortcuts and want to use them in VS, this extension brings the most popular Sublime Text keyboard shortcuts to Visual Studio code.

Key Features:

  • Extension officially released by Microsoft to import keyboard shortcuts
  • Imports the popular Sublime Text keyboards shortcuts used in programming
  • Brings keyboard shortcuts used on Windows, Linux and Mac
  1. HTML CSS Class Completion

HTML CSS Class Completion

As awesome as CSS is, it’s hard to remember all CSS classes. This extension auto-completes CSS class names as you type, based on the CSS files in your workspace.

Key Features:

  • Customize the directories where the extension will search for files
  • Provides manual re-caching for class definition auto completion
  • Provides support for PHP, Razor, and Jade
  1. View in Browser

View in Browser

If you like viewing your HTML in a browser frequently for feedback as you code, this Visual Studio code extension will help you do it. It renders HTML files in your system’s default browser.

Key Features:

  • Provides two options to edit, one from context menu and another from command
  • Set your default browser according to your preference
  • Supports Mozilla, Chrome, Internet Explorer and Safari
  1. NgBootstrap Snippets

NgBootstrap Snippets

Get NgBootstrap snippets, which consist of Angular and Bootstrap 4 snippets, with this helpful extension.

Key Features:

  • Typing ngb gives a quick list of the snippets available
  • Unfolds snippets as soon as you press enter
  • Provides alerts, dropdown, progressbar and many other snippets
  1. jQuery Code Snippets

jQuery Code Snippets

jQuery is a powerful JavaScript library. You won’t remember every single function, and with this extension, you don’t need to. jQuery Code Snippets brings a vast library of over 130 jQuery Code snippets to Visual Studio code.

Key Features:

  • Provides a quick list of all the snippets after typing ‘jq’
  • Sets single or multiple attributes from the matched elements
  • Snippets have been added from the 2015 Visual Studio jQuery Snippets
  1. Bower

Bower

Bower makes building apps easy with its package management capabilities, providing access to most Bower commands within VS Code. With the Bower Visual Studio Code extension, you can install, uninstall, search for, and update packages effortlessly.

Key Features:

  • Interactive Bower command prompts have been built in the IDE
  • Write, install and uninstall bower.json and other Bower packs
  • Find files according to the files you are editing
  1. Git History

Git History

Git is the most popular version control system today. It enables distributed version control for modern web apps. Sometimes, you want to be able to re-trace your code to its previous versions and restore that version. The Git History extension gives you the complete Git history, including a history of inline files, all within Visual Studio Code.

Key Features:

  • Shows history in graphs and charts according to your preference
  • Get multiple history graphs to create a comparison
  • Keyboard shortcuts are provided for quick and easy history viewing
  1. Angular Files

Angular Files

Scaffolding can be an important first step when building an app, but it can be complex. This Visual Studio extension lets you quickly scaffold Angular file templates.

Key Features:

  • Supports non Angular-cli file structure
  • Also supports angular-cli.json with different style extensions
  • Provides good support for app structure
  1. Auto-Open Markdown Preview

Auto-Open Markdown Preview

Previewing markdown files in a parallel window can be a pain due to the need to repeatedly type ‘Ctrl + K V’ or ‘Cmd + KV’. With the help of this Visual Studio Code extension, you can get automatic previews of the Markdown files you open.

Key Features:

  • Latest version supports Visual Studio code 1.10.0 and higher
  • Removes the need to press ‘Ctrl + K V’ for a preview of the Markdown file
  • Shows previews next to the Markdown file you open
  1. Angular v4 TypeScript Snippets

 

Angular v4 TypeScript Snippets

Save time when coding using the snippets you get with this Visual Studio code extension, which adds snippets for Angular for TypeScript and HTML.

Key Features:

  • Works with Visual Studio 0.10.1 and higher
  • Has been updated for Angular v4.0.0 release
  • Keyboard shortcuts to activate snippets from within the editor
  1. ESLint

ESLint

ESLint enables better code quality by bringing the ESLint library to Visual Studio Code. It also allows you to identify and act on patterns in JavaScript.

Key Features:

  • Uses ESLint library from the opened workspace folder
  • Documentation for ESLint is provided for new ESLint users
  • Searches global version of the file if the library doesn’t have one
  1. Bracket Pair Colorizer

Bracket Pair Colorizer

If different brackets confuse you, this Visual Studio code extension will help you match brackets with color-coding options. You can define which characters to match as well as preferred colors.

Key Features:

  • Allows custom characters in custom brackets to be configured
  • List of various colors available even for orphaned brackets
  • New brackets can be added or removed from the list
  1. MsSQL

mssql

MySQL is a very helpful database tool. This extension helps you develop and connect various systems to the database you use, which helps your team access data more efficiently.

Key Features:

  • Customizable command shortcuts and extension options
  • Allows you to create and manage various connection profiles
  • Connects to SQL Data warehouse, Azure SQL Database, and Microsoft SQL Server
  • Evaluate scripts and view results in a grid
  • Save results to JSON or .csv file format to see in the editor
  1. VScode-Icons

VS Code Icons

If the simplistic Visual Studio does not appeal to you, this extension adds icons to VS Code, making it more visually appealing and fun to work in. This extension was originally created to enable icons before VS Code began supporting them for files and folders. Now that VS Code does support icons, VScode-Icons aims to provide the most comprehensive set of icons available.

Key Features:

  • Provides a complete set of icons to choose from in VS Code
  • User can make icons look different and change the extension association at will
  • Allows suggestions to add to the growing icon list
  1. Angular 2+ Snippets

Angular 2+

For programmers using the new Angular 2+ and need a supporting snippet extension, this Visual Studio extension is the one you need.

Key Features:

  • Provides about 89 Angular 2+ snippets (and growing)
  • Snippets are updated for versions 2.4.11 and 4.0.0
  • Snippets of TypeScript, HTML, ngRx and code examples are available
  • Suggests Angular 2 solutions when Angular 1 keywords are entered
  1. TypeScript Hero

TypeScript Hero

TypeScript lets you check and refactor JavaScript code as you type. TypeScript Hero brings this powerful tool to Visual Studio Code, automatically importing your project and libraries to your current file.

Key Features:

  • Adds every missing import in one command
  • Fixes coding errors with a feature called ‘Light bulb.’
  • Sort and organize your imports and remove unused imports
  1. Path Intellisense

Path Intellisense

If you have a hard time remembering the path of a file, Path Intellisense can help. It auto-completes filenames as you work, saving you both time and energy.

Key Features:

  • Autocomplete feature fills in the file path as you type
  • Hidden files can be made visible with a simple setting
  • Names with dashes are also shown in autocomplete
  1. HTMLHint

HTMLHint

HTMLHint is a static code analysis tool that can be used with an IDE to improve the quality of code. If you’re looking to use HTMLHint in Visual Studio Code, there’s an extension for that.

Key Features:

  • Highlights errors in the HTML file with squiggles
  • Reports numbers of errors on status bar for easy viewing
  • Provides options to set and define rules for checking
  1. Bookmarks

Bookmarks

Tired of endlessly scrolling through your code, or using a ‘Ctrl + F’ to find that exact line of code you need to fix? This Visual Studio code Bookmarks extension makes creating and jumping through code easy and smooth.

Key Features:

  • Set bookmarks anywhere in your code for easy access later
  • Autocomplete feature makes navigating to your bookmarks easy
  • Bookmarks are saved on your current project in sessions
  • Bookmarks are restored on reopening the file
  1. Office UI Fabric Snippets

Office UI Fabric Snippets

Get the Office UI Fabric Snippets support to create HTML document with this Visual Studio code extension. It has many UI elements that can be easily added to your code.

Key Features:

  • Supports UI Fabric 3.0.0 Beta 2
  • Lists available after typing Fabric while coding
  • Provides overall change to the HTML document
  • Breadcrumbs, buttons, callouts, date picker and label are some of many snippets available
  1. Haskelly

Haskelly

Program using Haskell causally or as an expert using this Visual Studio extension that enables support for Haskell development.

Key Features:

  • Supports Haskell and Cabal (.hs and .cabal)
  • Code snippets for structures and popular functions
  • Provides complete all round support for Haskell developers
  • Customizable according to the preference of the user
  1. Angular UI Bootstrap Snippets

Angular UI Bootstrap Snippets

Use snippets of UI Bootstrap to code your Angular project using this Visual Studio Code extension. In HTML or JavaScript files, simply start typing uib- and Angular UI Bootstrap Snippets provides autocompletions.

Key Features:

  • Autocomplete feature to make finding snippets easy
  • Quickly toggle through different highlighted placeholder by pressing tab
  • Can be added to any JavaScript or HTML file
  1. Angular 1 JavaScript and Typescript Snippets

Angular 1 Snippets

You may be happy with things that just work, and don’t feel the urge to keep updating to the latest version of a framework. If that sounds like you, and you’re still using Angular 1, you’ll love this extension. It provides snippets for Angular 1 for both JavaScript and TypeScript.

Key Features:

  • Provides snippets for JavaScript and TypeScript
  • Provides snippets for users of Angular 1
  • Quick and easy search for snippets
  • Keyboard shortcuts to streamline your workflow
  1. Language Support for Java(TM) by RedHat

Language Support for Java

Many enterprise applications rely on Java. If you’re having trouble with developing or editing in Java in Visual Studio Code, this VS code extension will make your life easier by providing language support via Eclipse JDT Language Server, which utilizes Eclipse JDT, M2Eclipse and Buildship.

Key Features:

  • Provides Java support through Eclipse JDT server language
  • Shows status of language tools on lower right corner
  • Provides details about failures in language coding
  • Supports Maven, Gradle and Eclipse
  • Code completion, code navigation, highlights, formatting, snippets and more
  1. Azure Functions Tools

Azure Functions Tools

Microsoft Azure is one of the most popular cloud services used by developers and Ops teams to host and manage applications. Azure Functions is a powerful tool that lets you implement code based on event triggers, and the Azure Functions extension makes this possible in Visual Studio Code.

Key Features:

  • Includes JSON IntelliSense for host and function
  • Provides shortcut to quickly activate Intellisense
  • Works on Visual Studio code 1.11.0 and higher
  1. Cordova Tools

Cordova Tools

Apache Cordova is a popular mobile development framework. This Cordova extension assists in integrating commands, code-hinting and debugging for Apache Cordova within VS Code.

Key Features:

  • Command palette finds errors in code and debugs them
  • Uses Intellisense to work around plugin APIs
  • Works smoothly with any project based on Cordova
  • Added support for the Ionic framework
  1. Vim

Vim

A favorite text editor of many programmers, Vim is now emulated in Visual Studio Code. This Vim extension makes writing and editing code easier.

Key Features:

  • Multiple code commands combinations
  • Possess a versatile command remapping capability
  • Multiple Vim commands can be followed by multiple cursors
  1. React Native Tools

React Native Tools

React Native is the popular mobile development framework open sourced by Facebook. This Reach Native extension lets you edit, develop, debug and integrate commands for React Native, providing a development environment for React Native projects.

Key Features:

  • Creates an environment to support mobile project development
  • Use command palette to debug and quickly run commands
  • Use IntelliSense to browse parameters for React Native APIs
  1. PHP Debug

PHP Debug

If you’re programming with PHP and are looking for an extension to help you debug your PHP code, PHP Debug is the VS Code extension you need, offering a variety of features to streamline development and ensure bug-free code.

Key Features:

  • Acts as an adapter between VS code and XDebug
  • Connects to remote debuggers as well
  • Allows user to set breakpoints according to their preference (line, conditional, and function breakpoints)
  • Break on entry or on uncaught exceptions and errors / warnings / notices
  • Supports multiple, parallel requests
  • Runs as CLI or without debugging
  1. Code Runner

Code Runner

Code Runner is an extension that enables you to run any language’s code snippets in Visual Studio Code, with support for every popular programming language including both legacy languages and those that have gained popularity in recent years such as Clojure, Objective-C, Rust, Racket, AutoHotkey, AutoIt, and many others.

Key Features:

  • Provides an Output Window to check the output of a snippet
  • Supports a wide range of programming languages like C++, PHP, Clojure, Racket, Python and much more
  • Custom commands
  1. vscode-random

vscode-random

Generating and adding random data directly into your project, this Visual Studio extension does exactly what its name suggests. It uses the ChangeJS library and was inspired by the Random extension for Atom.

Key Features:

  • Configures names, years, numbers, etc.
  • Multiple commands can be accessed and worked
  • Quick install and launch option
  1. Travis CI Status

Travis CI Status

Apart from Jenkins, Travis is a widely known and used CI platform. Checking the build status of your code during the build stage is key. This Visual Studio code extension enables you to check your project’s Travis CI status without leaving VS Code.

Key Features:

  • Small status indicator present in the status bar for convenience
  • Update status and open in Travis are two default commands
  • Uses a combination of repository/username to find test
  1. htmltagwrap

htmltagwrap

This simple, useful Visual Studio code extension provides you with the option of wrapping the codes you select in HTML brackets.

Key Features:

  • Wraps single or multiple inline selections
  • Quick wrap shortcuts available
  • Supports spaces and tabs for indentation
  1. Git Blame

Git Blame

If you’re looking for a tool to help you see the Git Blame to view the revision history for your code, this Visual Studio code extension does just that. It appears in your status bar for easy access.

Key Features:

  • Shows git blame information for selected lines
  • Git blame information pings in the status bar
  • Reduced text size for visibility
  1. Reactjs Code Snippets

Reactjs Code Snippets

Reactjs is a popular open source UI framework released by Facebook. For the developers using Reactjs, this Visual Studio code extension provides snippets in ES6 syntax.

Key Features:

  • Supports languages like JavaScript, TypeScript and React
  • Based on the bable-sublime-snippet package
  • View already-installed snippets or add new snippets
  1. Settings Sync

Settings Sync

Github Gist is a nifty way to share code and other pieces of text. With the Settings Sync extension, you can sync snippets, themes and other extensions across different machines using Gist.

Key Features:

  • Single click upload and download
  • Supports anonymous, Gist and Github account token to log in
  • Synchronizes settings, snippets, extensions and themes
  1. Project Manager

Project Manager

Switching between projects can become tiresome, slowing down your progress. This Visual Studio code extension makes the navigation process seamless, allowing you to multitask like a pro.

Key Features:

  • Makes projects within Visual Studio easily accessible, and easy to switch between
  • Scans projects and differentiates them according to code used
  • Automatically caches the projects after scanning

These are a few of the many more extensions available in the Visual Studio Marketplace. Each extension serves a particular purpose in helping you program. They perform a variety of tasks like enabling you to add snippets, navigate through projects, use different programming language and add other features. We hope this list helps you and you find the extension you have been searching for and opens up your mind to the ways you can extend Visual Studio Code to simplify and supercharge your development.

What is NullReferenceException? Object reference not set to an instance of an object

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Null reference errors are responsible for a good percentage of all application bugs. They are usually very simple problems caused by not adding additional logic to ensure that objects have valid values before using them.

The following code will throw a NullReferenceException if the variable “text” being passed in is null. You can’t call ToUpper() on a null string.

public void MyMethod(string text)
{
     //Throws exception if text == null
     if (text.ToUpper() == "Hello World")
     {
          //do something
     }
}

You can also have null reference exceptions because any type of object is null. For example, in the code below, the SqlCommand object is never initialized. Not running a SQL query would be a serious problem for your application. A null string might be something you just ignore and move on. Other times, like with the SqlCommand, it could be a fatal issue you don’t want to ignore.

SqlCommand command = null;
//Exception! Object reference not set to an instance of an object
command.ExecuteNonQuery();

Use the Null Conditional Operator to Avoid NullReferenceExceptions

One of the best new additions to C# was the null conditional operator. Instead of having a crazy amount of “variable != null” type checks, you can use the “?” and your code will short circuit and return null instead of throwing the exception. This will make more sense with some examples below:

text?.ToUpper(); //from previous example, would return null

int? length = customerList?.Length; // null if customerList is null   
Customer first = customerList?[0];  // null if customerList is null  
int? count = customerList?[0]?.Orders?.Count();  // null if customerList, the first customer, or Orders is null  

Use Null Coalescing to Avoid NullReferenceExceptions

Another great feature is null coalescing, which is the “??” operator. It works great for providing a default value for a variable that is null. It works with all nullable datatypes.

The following code throws an exception without the null coalescing. Adding “?? new List<string>()” prevents the “Object reference not set to an instance of an object” exception.

List<string> values = null;

foreach (var value in values ?? new List<string>())
{
    Console.WriteLine(value);
}  

Simple Examples of Null Values Causing Problems

Some of the most common causes are settings, database calls, or API type calls not returning expected values. For example, you add a new field to your database and don’t populate default values for every record. Randomly records get queried and the code didn’t account for that new field being null. KA-BOOM: Object reference not set to an instance of an object.

The Golden Rule of Programming

For years I have had a saying that I say to my team all the time. I call it the golden rule of programming.  I think every new programmer needs a tattoo that says it.

“If it can be null, it will be null”

The good news is that a lot of null reference errors can be avoided by adding additional logic and code to ensure objects are not null before trying to use them. Developers should always assume that everything is invalid and be very defensive in their code. Pretend every database call is going to fail, every field is going to have messed up data in it. Good exception handling best practices are critical.

Tips to Prevent Null Reference Exceptions

  1. Initialize variables with valid values.
  2. If a variable can be null, then check for null and handle it appropriately
  3. Use the ? operator on methods when possible. stringvar?.ToUpper();
  4. Use tools like Resharper to help point out potential null reference exceptions

Conclusion

Null reference exceptions are a very common problem in .NET and all programming languages. Luckily, we can all blame Tony Hoare. He invented null references and even calls it the billion dollar mistake.

If you follow my golden rule, I promise you will have far fewer bugs in your apps.  If it can be null, it will be null!

Understanding # of Exceps Thrown / Sec, and How to Find Exceptions

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Exceptions can cause big performance problems. One of the most important performance metrics to track about your application is “# of Exceps Thrown / Sec“. It is available via Windows Performance Counters under the category .NET CLR Exceptions. In this article we will talk about exceptions, how to monitor your exception rate and find the actual exceptions being thrown.

Is # of Exceps Thrown / Sec High? Should You Worry?

It is common for most applications to throw some exceptions. Don’t worry if this counter is not zero. It is important to understand what is normal for your specific application and what the exceptions are. Also, remember that this counter includes all thrown exceptions, that does not mean they are all unhandled exceptions that cause big problems for your users.

For example, our applications commonly throw exceptions due to transient type issues connecting to Azure resources, external web services, etc. We also throw a lot of exceptions trying to parse possible JSON strings. We use retry logic and good exception handling best practices in our code to handle these types of normal problems.

Using our app as an example, we expect a certain amount of “noise” in our # of Exceps Throw / Sec and you should too. For us, 5-10 exceptions per second is nothing to be alarmed about in some of our apps.

I have seen applications throw 100 exceptions per second. That is a serious problem!

I highly recommend tracking # of Exceps Throw / Sec within your monitoring system so your understand what is normal for your application and can identify spikes or changes.

MORE: How GWB Found Hidden Exceptions and Application Performance Problems

How to View All Thrown Exceptions

If your # of Exceps Throw / Sec seems like a high number and your application logging doesn’t show any exceptions, the exceptions are being handled but not logged somewhere in your code. Your application is “swallowing” or “throwing away” the exceptions

Luckily, .NET provides an easy mechanism to view all exceptions that are being thrown. You can see every exception, no matter why it is thrown or how it is handled. This functionality is called first chance exceptions.

Our blog post about C# exception handling best practices does a great job of covering first chance exceptions and a few ways to find hidden exceptions in your code.

If you can easily modify your code, I would recommend subscribing to the .NET even for first chance exceptions. You can then log all of the exceptions however you prefer.

AppDomain.CurrentDomain.FirstChanceException += (sender, eventArgs) =>
{
        //log.Debug ...
};

View All Exceptions With Retrace

The easiest thing to do is sign up for a 14 day free trial of Retrace and install it on your server. By default, Retrace only collects unhandled exceptions to avoid collecting a lot of noise. Within the environment settings, you can set it to collect all thrown exceptions.

Retrace will show you the exact exceptions that are being thrown, the stack traces, and uniquely identify all your exceptions.

MORE: Advanced Error Monitoring, Tracking & Reporting with Retrace

How to Monitor # of Exceps Thrown / Sec

One of the great features of Retrace is smart defaults of monitoring key application metrics like # of Exceps Thrown / Sec. If you are not using Retrace, check if your server or application monitoring tools can monitor Windows Performance Counters and set it up to do so.

Screenshot of Retrace Monitoring of # of Exceps Thrown / Sec

Screenshot of Retrace Monitoring of # of Exceps Thrown / Sec

How to Generate C# Random Numbers, Pseudo vs Secure Random Numbers

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Random numbers are used in a wide variety of software applications. Depending on what you are using random numbers for, you will need to decide which type to use. For a music jukebox, the accuracy is not very critical. For something like a lottery or slot machine, the random number generator must be extremely accurate.

There are two types of random number generators in C#:

  • Pseudo-random numbers (System.Random)
  • Secure random numbers (System.Security.Cryptography.RNGCryptoServiceProvider)

Pseudo vs Secure Random Numbers

The key difference is the chance that the seed value used to do the randomization may not be changing quickly and randomly enough. For example, System.Random relies on the computer system clock. If multiple Random() objects were created at the exact same time, they could create the same sequence of random numbers.

Secure random numbers are called “secure” because of potential security vulnerabilities in weak random number generators. If a hacker could figure out a pattern to your random crypto keys, they may be able to increase their chances of hacking in.

MORE: “True” vs. pseudo-random numbers (Wikipedia)

How to use C# System.Random Number Generator

System.Random works great for basic use cases of creating random numbers.

For example, we use it at Stackify to randomly schedule some background jobs so they don’t all run exactly at the same second and minute every hour. We want to randomly distribute the jobs over the course of an hour. For this type of use case, System.Random works fine for us.

Examples of using System.Random to generate C# random numbers:

Random random = new System.Random();
int value = random.Next(0, 100); //returns integer of 0-100
double value2 = random.NextDouble(); //returns floating point 0.0-1.0

var byteArray = new byte[256];
random.NextBytes(byteArray); //fill with random bytes

 

How to Create Secure C# Random Numbers With RNGCryptoServiceProvider

Secure random numbers are important if you are creating a very high volume of numbers or need to ensure proper uniqueness over a long period of time. If you were generating random numbers for a game like Roulette, it would be important that the random numbers were properly distributed over all of the potential outcomes (0-36) properly.

RNGCryptoServiceProvider provides a random set of bytes. You must then convert those bytes to the data type that you need.

RNGCryptoServiceProvider provider = new RNGCryptoServiceProvider();
var byteArray = new byte[4];
provider.GetBytes(byteArray);

//convert 4 bytes to an integer
var randomInteger = BitConverter.ToUInt32(byteArray, 0);

var byteArray2 = new byte[8];
provider.GetBytes(byteArray2);

//convert 8 bytes to a double
var randomDouble = BitConverter.ToDouble(byteArray2, 0);

 

How to Convert C# String to Int

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

The .NET Framework provides a couple built in ways to convert a C# string to int, or several other numeric datatypes. It is important that you do so in a way that does not throw potential exceptions.

Unsafe Ways to Convert a C# String to Int

You can use a couple built in methods, as shown below, to convert a string to int. Both of these would throw an exception if the string value is not a valid integer.

int.Parse((string) null); // throws exception
int.Parse("Hello"); // throws exception
int.Parse("1.5"); // throws exception
var i = int.Parse("1"); // Works!

var i2 = Convert.ToInt32((string) null); //No exception! returns 0
Convert.ToInt32("hello"); //throws exception
Convert.ToInt32("1.5"); //throws exception
Convert.ToInt32("1"); // Works!

The built-in methods Convert.ToInt32() and int.Parse() produce the same results, except for the null string scenario. If we peek under the covers to see what the source code for Convert.ToInt32() does, you can see it has special logic to look for null and calls int.Parse internally.

public static int ToInt32(string value)
{
    if (value == null)
       return 0;
    return int.Parse(value, (IFormatProvider) CultureInfo.CurrentCulture);
}

How to Prevent Exceptions When Converting a C# String to Int

Bad data is inevitable. You should definitely handle potential exceptions when parsing data and converting datatypes. In this example, possible exceptions converting a string to int.

Luckily, the .NET Framework provides int.TryParse which is designed for this exact problem. Below is an example of how to properly use it.

string incomingData = "Hello";
int parsedResult; 
int.TryParse(incomingData, out parsedResult);

The only problem is that the variable parsedResult will end up being a 0 if the TryParse fails. The data being parsed could also be a 0. If you need to know the difference between the incomingData being 0 and just defaulting to 0, you will need to actually use the boolean result of TryParse to know if it actually parsed or not.

string incomingData = "Hello";
int parsedResult; 

if (int.TryParse(incomingData, out parsedResult))
{
   //do something with the data. 
}

Make Some Helper Methods to Simplify Converting a C# String to Int

A good way to simplify your TryParse code is to make your own helper methods via C# extension methods. You could make extension methods to support different default values or nullable integers.

public static class IntegerExtensions
{
    public static int ParseInt(this string value, int defaultIntValue = 0)
    {
        int parsedInt;
        if (int.TryParse(value, out parsedInt))
        {
            return parsedInt;
        }

        return defaultIntValue;
    }

    public static int? ParseNullableInt(this string value)
    {
        if (string.IsNullOrEmpty(value))
            return null;

        return value.ParseInt();
    }
}

You could then use these helper methods in your code to work with a default other than 0, or a nullable integer.

int number = myString.ParseInt(); // returns value or 0
int number2 = myString2.ParseInt(-1); // with default value -1
int? number3 = myString3.ParseNullableInt(); // returns value or null

Conclusion

Converting a string to int is a common task that developers do almost daily. This guide should help you avoid exceptions in your code! Check out our guide on C# exception handling best practices to learn more about exception handling.

What is an Unhandled Exception, and How to Catch All C# Exceptions

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

What is an Unhandled Exception?

An exception is a known type of error. An unhandled exception occurs when the application code does not properly handle exceptions.

For example, When you try to open a file on disk, it is a common problem for the file to not exist. The .NET Framework will then throw a FileNotFoundException. This is a simple example of a potential known problem that is accounted for within the code.

An unhandled exception occurs when a developer does not anticipate and handle a potential exception

Let’s take this code sample below. The developer is assuming that within “args” a valid file path will be passed in. The code then loads the contents of the file path being passed in. This code will throw exceptions if no file path is passed in or the file does not exist. This would cause unhandled exceptions.

static void Main(string[] args)
{
    string fileContents = File.ReadAllText(args[0]);

    //do something with the file contents
}

This code can easily throw several types of exceptions and lacks exception handling best practices.

MORE: C# Exception Handling Best Practices

How to Catch All C# Exceptions

The .NET Framework provides a couple events that can be used to catch unhandled exceptions. You only need to register for these events once in your code when your application starts up. For ASP.NET, you would do this in the Startup class or Global.asax. For Windows applications, it could be the first couple lines of code in the Main() method.

static void Main(string[] args)
{
  Application.ThreadException += new ThreadExceptionEventHandler(Application_ThreadException);
  AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);

  string fileContents = File.ReadAllText(args[0]);
  //do something with the file contents
}

static void Application_ThreadException(object sender, ThreadExceptionEventArgs e)
{
  // Log the exception, display it, etc
  Debug.WriteLine(e.Exception.Message);
}

static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
  // Log the exception, display it, etc
  Debug.WriteLine((e.ExceptionObject as Exception).Message);
}

MORE: AppDomain.UnhandledException Event (MSDN)

View Unhandled Exceptions in Windows Event Viewer

If your application has unhandled exceptions, that may be logged in the Windows Event Viewer under the category of “Application”. This can be helpful if you can’t figure out why your application suddenly crashes.

Windows Event Viewer may log 2 different entries for the same exception. One with a  .NET Runtime error and another more generic Windows Application Error.

From the .NET Runtime:

Application: Log4netTutorial.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.IndexOutOfRangeException
   at Log4netTutorial.Program.Main(System.String[])

Logged under Application Error:

Faulting application name: Log4netTutorial.exe, version: 1.0.0.0, time stamp: 0x58f0ea6b
Faulting module name: KERNELBASE.dll, version: 10.0.14393.953, time stamp: 0x58ba586d
Exception code: 0xe0434352
Fault offset: 0x000da882
Faulting process id: 0x4c94
Faulting application start time: 0x01d2b533b3d60c50
Faulting application path: C:\Users\matt\Documents\Visual Studio 2015\Projects\Log4netTutorial\bin\Debug\Log4netTutorial.exe
Faulting module path: C:\WINDOWS\System32\KERNELBASE.dll
Report Id: 86c5f5b9-9d0f-4fc4-a860-9457b90c2068
Faulting package full name: 
Faulting package-relative application ID: 

Find Unhandled Exceptions with Retrace

One of the great features of Retrace is it’s error monitoring capabilities. Retrace can automatically collect all .NET exceptions that are occurring within your application. This includes unhandled exceptions but can also include all thrown exceptions, or first chance exceptions.

 

How to Deploy ASP.NET Core to IIS & How ASP.NET Core Hosting Works

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

Previously, I discussed the differences between Kestrel vs IIS. In this article, we will review how to deploy an ASP.NET Core application to IIS. Deploying an ASP.NET Core app to IIS isn’t complicated, but ASP.NET Core hosting is a little different than ASP.NET.

How to Configure Your ASP.NET Core App For IIS

The first thing you will notice when creating a new ASP.NET Core project is they are actually console applications. Your project now contains a Program.cs file just like a console app would have and it contains the following code:

 
public class Program
{
    public static void Main(string[] args)
    {
        var host = new WebHostBuilder()
            .UseKestrel()
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseIISIntegration()
            .UseStartup()
            .Build();

        host.Run();
    }
}

What is the WebHostBuilder?

All ASP.NET Core applications require a WebHost object that essentially serves as the application and web server. WebHostBuilder is used to configure and create the WebHost. You will normally see UseKestrel() and UseIISIntegration() in the WebHostBuilder setup code.

What do these do?

  • UseKestrel() – This registers the IServer interface for Kestrel as the server that will be used to host your application. In the future, there could be other options, including WebListener which will be Windows only.
  • UseIISIntegration() – This tells ASP.NET that IIS will be working as a reverse proxy in front of Kestrel. This then specifies some settings around which port Kestrel should listen on, forwarding headers, and other details.

If you are planning to deploy your application to IIS, UseIISIntegration() is required

What is AspNetCoreModule?

You may have noticed that ASP.NET Core projects create a web.config file. This is only used when deploying your application to IIS. It registers the AspNetCoreModule as an HTTP handler.

Default web.config for ASP.NET Core:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <system.webServer>
    <handlers>
      <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified"/>
    </handlers>
    <aspNetCore processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" stdoutLogEnabled="false" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="false"/>
  </system.webServer>
</configuration>

AspNetCoreModule handles all incoming traffic to IIS and acts as the reverse proxy that knows how to hand the traffic off to your ASP.NET Core application. You can view the source code of it on GitHub. It also ensures that your web application is running. It is responsible for starting your process up.

Steps to Deploy ASP.NET Core to IIS

Before you deploy, you need to make sure that WebHostBuilder is configured properly to use Kestrel and IIS. Your web.config should also exist and look similar to our example above.

Step 1: Publish to a File Folder

Publish to Folder With Visual Studio 2017

Step 2: Copy Files to Preferred IIS Location

Now you need to copy your publish output to where you want the files to live. If you are deploying to a remote server, you may want to zip up the files and move to the server. If you are deploying to a local dev box, you can copy them locally.

For my example, I am copying the files to C:\inetpub\wwwroot\AspNetCore46

You will notice that with ASP.NET core there is no bin folder and it potentially copies over a ton of different .NET dlls. Your application may also be an EXE file if you are targeting the full .NET Framework. My little sample project had over 100 dlls in the output.

Step 3: Create Application in IIS

First, create a new IIS Application Pool. You will want to create one under the .NET CLR version of “No Managed Code“. Since IIS only works as a reverse proxy, it isn’t actually executing any .NET code.

Second, create your new application under your existing IIS Site, or create a new IIS site. Either way, you will want to pick your new IIS Application Pool and point it at the folder you copied your ASP.NET publish output files to.

Step 4: Load Your App!

At this point, your application should load just fine. If it does not, check the output logging from it. Within your web.config file you define how IIS starts up your ASP.NET Core process. Enable output logging by setting stdoutLogEnabled=true and you may also want to change the log output location as configured in stdoutLogFile. Check out the example web.config before to see where they are set.

Advantages of Using IIS with ASP.NET Core Hosting

Microsoft recommends using IIS with any public facing site for ASP.NET core hosting. IIS provides additional levels of configurability, management, security, logging, and many other things. Check out my blog post about Kestrel vs IIS to see a whole matrix of feature differences. It goes more in depth about what Kestrel is and why you need both Kestrel & IIS.

One of the big advantages to using IIS is the process management. IIS will automatically start your app and potentially restart it if a crash were to occur. If you were running your ASP.NET Core app as a Windows Service or console app, you would not have that safety net there to start and monitor the process for you.

How to Deploy ASP.NET Core Web App to Azure & How to Use Retrace With ASP.NET Core

Barry Luijbregts Developer Tips, Tricks & Resources, Insights for Dev Managers, Live Queue Leave a Comment

ASP.NET Core is the latest version of ASP.NET. It is built from the ground up using .NET Core, the lightweight, composable new .NET Framework. Some people love it, some don’t. However, this is the future. Microsoft will move forward with .NET Core and in the future, the regular, full-fledged versions will go away.

I like ASP.NET core a lot. It has a ton of new features, is much more streamlined and it is fast, really fast. And, of course, it runs on Azure.

In this post, I’m going to show you how to deploy a simple ASP.NET Core application to Azure. I’ll also show you how to instrument the application with Stackify Retrace, which is an Application Performance Management tool.

In this post, I’m going to use an example application called SimplCommerce. This is an open source e-commerce application, based on ASP.NET Core, hosted on GitHub. Basically, the application consists of a web application and a database.

I’ll guide you through the following steps:

  • Create the SimplCommerce Database on SQL Azure
  • Configuring the Application to Use Stackify Retrace
  • Deploying the Application to Azure
  • Activating Stackify Retrace for the Web App
  • Stackify Retrace Benefits in Action

Create the SimplCommerce Database on SQL Azure

The SimplCommerce application uses a SQL database to store its product catalog and other data. To start using the application, you first need to setup a database. You can do this on your local machine, or somewhere else.

My plan is to deploy the whole application to Azure. When I work in Azure, I’d like to work with a database that runs there as well, as that provides me with confidence that whatever I’m doing runs in Azure SQL and I’m not going to run into any weird differences with an on-premises SQL Server.

To setup up an Azure SQL Database and connect to it, we need to execute the following steps:

  1. Create an Azure SQL Database
  2. Change the connectionstring of the application
  3. Open the firewall to allow my local machine to connect to the Azure SQL Server

Step 1: Create an Azure SQL Database

Step one is to create an Azure SQL Database to host the SimplCommerce data. I do this by creating new database in the Azure Portal. This takes me through a wizard that lets me create a database, an accompanying Azure SQL Server and configure them.

As you can see in the image above, I chose the basic pricing tier, as this is just a test application and won’t need a lot of horsepower.

That’s it, Azure SQL server and database created. Now to connect to it. I like to connect to the Azure SQL Database from my local machine and use it for local development. Although this seems slow because the database is somewhere else than my code, it really is fast enough to work with.

Step 2: Change the connectionstring of the Application

To connect the SimplCommerce application to the database, I need to change its connectionstring. In ASP.NET Core, this is located in the appsettings.json file and underlying file(s). These are .json files, that contain configuration, just like the web.config did in the past.

The beauty of these is that they work, based on the environment variable that you set. For instance, you can create an appsettings.development.json file for when the environment variable is development, like this:

The environment variable that I’m talking about is the ASPNETCORE_ENVIRONMENT variable. By default, when you debug in Visual Studio, ASP.NET Core uses this variable with the value ‘development’. When you deploy your application, by default, the variable will have the value ‘production’. You can overwrite this variable in whatever hosting environment you use or even in the settings of the projectfile in Visual Studio.

Anyway, to connect to the database from my local machine, I have put the connectionstring to the database in the appsettings.development.json file.

Step 3: Open the firewall to allow my local machine to connect to the Azure SQL Server

But that’s not enough. By default, Azure SQL Server has a firewall enabled, blocking all IP-addresses, except addresses within Azure, from accessing the server. Therefore, I need to add my IP-address to the firewall, so that I can reach the server. You can do that from the Azure Portal, in the settings of the Azure SQL Server.

That should do it. However, the database is empty and doesn’t even have any tables in it. On the SimplCommerce GitHub page, you’ll find the steps that you need to execute to fill the database with tables and data. This mainly involves running an Entity Framework migration and running a SQL script to fill the database with data.

Configuring the Application to Use Stackify Retrace

Before I deploy the application to Azure, I want to get it ready to send data to Stackify Retrace, so that we can monitor the app and see how it’s doing. This is a very powerful monitoring tool that allows me to monitor things like:

  • Performance of the application and dependencies
  • Errors within the application
  • Availability of the application

That first one is really important. I want to see the complete performance of the application. This means that I also want to see how my dependencies, like my Azure SQL Database, are performing and how I can improve that.

And the ability to be alerted when an error occurs in the app and drill down into it is of vital importance. Without it, I wouldn’t even know that errors are happening until I hear about it from a user, which is bad. Also, Retrace allows me to dive deeply into the error so that I can trace back what caused it and fix it.

All of this is really simple to set up.

First, I add a nugget package called StackifyMiddleware which you can do in Visual Studio like in the image below or by typing the following into the package manager console window: Install-Package StackifyMiddleware

After that, you need to activate the Stackify middleware, which you can do in the class of the application. Here, in the Configure method, you need to type the line app.UseMiddleware<StackifyMiddleware.RequestTracerMiddleware>(); Be sure to insert this line before the app.UseMvc statement.

C:\Users\barry\AppData\Local\Microsoft\Windows\INetCacheContent.Word\2017-04-07_15-07-54.png

And that’s it. Now, we are ready to deploy the application to Azure.

Deploying ASP.NET Core to Azure

I want this application to run in Azure. To do that, I will create an Azure Web App and deploy the application to it. ASP.NET Core applications run in Azure Web Apps, just like any other applications. Bear in mind though that older ASP.NET Core applications (based on the project.json format), will not be supported in Azure past the end of May 2017.

I will deploy the application to Azure using Visual Studio. This is really simple. You just right-click the project file of the ASP.NET App and choose publish. After that, a wizard like this appears that guides you through the process. It will even create a new Web App for you if you didn’t have one already.

Deploy ASP.NET Core to Azure

After this has completed, I have a Web App, with the application running in it. But it is not working yet.

Activating Stackify Retrace for the Web App

First of all, I need to install an extension in order for Stackify Retrace to capture the data of the Web App. I can do this from the Azure Portal. Here, you can choose the extensions menu from the options of the Web App and choose the Stackify APM+ extension and install it. This will enable Stackify to capture metrics of the resources that run the Web App, like CPU, Memory and so on.

Make sure to restart the Web App after installing this extension.

Configure the Web App

The next thing that I need to do is to configure the Web App.

Remember that I’ve put the connectionstring to the database in the appsettings.development.json file? Because of that, the application can’t connect to the database, because by default, it doesn’t use the .development.json file when it is deployed. Therefore, I need to put a connectionstring value in the ApplicationSettings of the Web App. This will override the configuration setting for the database connection, allowing the application to connect to the database.

Also, I need to insert the Stackify.ApiKey and its value into the application settings. This will tell Stackify that this Web App belongs to my account and allows me to see the data for the app. You can find the value of the key in your Stackify account panel. If you don’t have an account yet, register for a 14 day trial.

It is also recommended to set the Stackify.AppName and Stackify.Environment properties. These will show up in the Stackify portal and will help you to identify your application.

You can read the full instruction on setting up Stackify Retrace in an Azure App Services here.

Stackify Retrace Benefits in Action

With Stackify Retrace running in the background, I can see some interesting information about the application. Stackify is a SaaS product, so you can see your data when you log in to https://stackify.com. Here, I have access to all sorts of dashboards and metrics, which I can customize to my heart’s content.

One of my favorite dashboards is the Performance dashboard. Here, I can see very easily what the performance of my app is and where my performance comes from. Retrace can tell what my dependencies are, and look into those. In this case, I’m calling into an Azure SQL Database, which, according to the graph, takes the most time. I can also see very quickly how many failed requests there are in the current time period.

Next to that, the error overview is very helpful. I’ve introduced an error in the application and let it run for a while to be able to show you this overview. This shows the amount of errors that are happening and what they are, including a short stacktrace.

You can drill down into the stacktrace to see everything about that error, including everything about the HTTP request, like the HTTP headers.

This is very powerful and allows you to get to the bottom of an error quickly so that you can resolve it.

Start your own free trial of Retrace: Start 14 Day Free Trial

What is regression testing?

What is Regression Testing? Definition, Benefits, and Challenges of Regression Testing

Angela Stringfellow Insights for Dev Managers, Live Queue Leave a Comment

We talked a bit about the Software Development Life Cycle (SDLC) in a recent post, but today, we’re going to dig a little deeper into one particular and crucial element in the testing phase, particularly for Agile development: regression testing.

Definition of Regression Testing

Regression testing refers to the process of testing a changed or updated computer program to make sure the older software features – which were previously developed and tested – still performs exactly as they did before. One way to think about software regression is to think about somebody who implements a new air conditioning system in their home only to find that while their new air conditioning system works as expected, the lights no longer work.

Regression testing will often involve running existing tests against the modified code to make sure that the new code did not break anything that worked before the update. Regression testing can eliminate much of the risk associated with software updates. In addition to running existing tests, testers might tweak existing tests by introducing different secondary conditions as variables.

The Importance of Regression Testing

With the increased popularity of the Agile development methodology, regression testing has taken on added importance. Many companies today adopt an iterative, Agile approach to software development. For example, the great many software as a service (SaaS) providers will regularly update their features or add new functionality to their offerings with each software update. To ensure their core product remains unaffected by new feature additions, these companies will perform regression testing.

Regression testing is a fundamental part of the software development lifecycle. ProtoTech Solutions illustrates the concept nicely with this graph:

Regression Testing

The Challenges of Regression Testing

While regression testing is a vital element of the QA process, there are a number of challenges it brings.

  • Time Consuming: Regression testing can take a lot of time to complete. Regression testing often involves running existing tests again so testers might not be overly enthused at having to re-run tests.
  • Complex: Another thing to consider here is that as products get updated, they can grow quite complex causing the lists of tests in your regression pack to grow to a huge amount.
  • Communicating Business Value: Regression testing ensures existing product features are still in working order. Communicating the value of regression testing to non-technical leaders within your business can be a difficult task. Executives want to see the product move forward and making a considerable time investment in regression testing to ensure existing functionality is working can be a hard sell.

Regression Testing Best Practices:

As you and your team perform regression testing, there are some best practices to bear in mind.

  • Regularly Update Your Regression Pack: A regression pack is a collection of test cases that are performed as each new software update is completed. The scripted tests included in a regression pack are created with the requirement specifications of older versions of the software in mind. Random or ad-hoc tests may also be included in the pack. It is a good idea to keep your regression pack up to date. Regression testing can be time-consuming, the last thing you need is to include tests that check whether an older feature which has been removed is still working.
  • Focus on Highly-Trafficked Paths: Highly-trafficked paths are the most frequent use cases for your application. They will include the basic functionality of your application and most popular features. You should know your core group of users and the typical features and interactions they are most reliant on. Your regression pack must include tests that ensure this core functionality is working as it should.
  • Re-Run Successful Test Cases: Tests that have previously identified bugs and defects are also worth including in your regression pack. Alternatively, tests that the program passes consistently are good candidates for archival.
  • Automate: Automated regression testing can make the process much more efficient. Running the same tests over and over again can result in testers becoming bored and losing motivation. The quality of their work might suffer as well as motivation dips. Automated regression testing can free up your testers to work on the trickier case specific tests. Automation software can handle the more tedious tests. Another consideration here is that testing software can be re-used so, even though there is an initial outlay, are making efficiency gains on an ongoing basis meaning your testing software will quickly demonstrate ROI.

What is the Difference Between Regression Testing and Retesting?

Regression testing should not be confused with retesting. Regression testing is performed to ensure updated code has not caused any existing functionality to break. Retesting, on the other hand, refers to tests that are performed when a test case has identified some defects. Once the defects have been fixed, the tests are performed again to ensure the issues have been resolved.

Retesting is a higher priority than regression testing because issues have already been identified. Regression testing is focused on identifying potential issues.

Regression Testing Tutorials and Tips

Because regression testing can involve varying types of tests, there’s no single, clear-cut method for performing a “regression test,” per say. However, there are plenty of valuable insights and informative tutorials for incorporating regression testing practices into your overall SDLC. Check out the following tutorials and guides for more insights on selecting test cases, creating and executing a regression testing plan, best practices, and more:

Azure vs. AWS Cloud Comparison

Azure vs. AWS: Which One is Better?

Angela Stringfellow Developer Tips, Tricks & Resources, Insights for Dev Managers, Live Queue Leave a Comment

We’ve used Azure for nearly five years at Stackify; in fact, we built Retrace with Azure in mind. But is Azure the right cloud for you? The battle between Azure and AWS is heated, so we decided to see how they measure up in this head-to-head comparison. (For a side-by-side, feature-by-feature comparison chart including Google Compute, check out this post.)

Customers love Amazon Web Services (AWS).  There is no doubt about that.  The segment earned a profit of more than $3 billion on revenues of more than $12 billion for the full year of 2016.  That’s an increase from only a little less than $8 billion in revenues the year before.

That gives AWS a run rate of $14 billion, which is similar to the run rate posted by Microsoft for its commercial cloud services.  While it does not mean that Microsoft’s Azure is on equal footing as AWS as far as customers go, it does show us that Azure is a worthy contender for the cloud service provider of choice.  This perception is boosted by Azure’s offerings, which can easily match those of AWS.

If anything, Amazon has the starting lead as it has been in the cloud computing services space for more than ten years.  Azure has only been a market player starting 2010.  That is not to say that AWS is better by default because Microsoft is a known powerhouse and it has the resources to create an outstanding product or service if it decides to.  And by all indications, Azure is one of those products that the software giant is relying on for its revenues.  So how does Azure compare with AWS?

The Essentials

Amazon’s AWS has a range of offerings that fall under IaaS, and each of these is categorized into four classes:

  • content delivery and storage,
  • compute,
  • networking, and
  • database.

No matter which IaaS offering you get, you will be using Amazon’s identity and security services such as AWS CloudHSM’s key storage service and Amazon’s own Active Directory.  Not only that, but AWS offerings also have a range of management tools that users can use, including AWS Config, AWS Cloudtrail, and Cloudwatch.

Azure, on the other hand, also has four classes of offerings:

  • Data management and databases,
  • compute,
  • networking, and
  • performance.

Security and management tools include Active Directory Federation Services, Azure Active Directory, Multi-Factor Auth, among others, as well as a range of integrations for Azure monitoring and performance tweaks.

Developer Love: Deploying Apps & PaaS

One of the biggest advantages of cloud computing is the simplicity of deploying an application. As a developer, I want to deploy my app to multiple servers without having to deal with the actual servers. Being able to take advantage of PaaS features like SQL databases, caching, queueing, NoSQL and other technologies are also a big deal. Developers can use services like Redis & Elasticsearch without having to figure out how to install and manage them.

Azure has multiple app deployment options for developers. Including App Services, Cloud Services, Service Fabric, Container Service, Functions, Batch, WebJobs and more. No matter what type of application you are developing, Microsoft has great tools in place to help deploy and scale it.

AWS offers similar solutions with Container Service, Elastic Beanstalk, Lambda, and Batch. AWS does not have as many options or features on the app hosting side. Microsoft has flexed their knowledge of developer tools to have a little bit of an advantage for hosting cloud apps.

Containers seem to be the preferred mechanism to deploy apps in the future, especially for open source applications. Look for more and more advancements around hosting containerized apps in the cloud.

Hybrid Cloud & Legacy Apps

One of the hindrances of companies that are planning to migrate to cloud computing is their use of legacy apps.  Not all companies would have the resources to create new apps for the cloud environment or even start everything up.  For those that need to rely on legacy apps, a hybrid cloud that would combine the cloud environment with their data centers would be a lot of help.

Hybrid clouds are also one of the most popular choices for some companies that do not want to make a full conversion to the cloud and would want to keep some of their data and systems in-house.

Hybrid clouds are easier with Azure, partly because Microsoft has foreseen the need for hybrid clouds early on.  Azure offers substantial support for hybrid clouds, where you can use your onsite servers to run your applications on the Azure Stack.  You can even set your compute resources to tap cloud-based resources when necessary. This makes moving to the cloud seamless.  Aside from that, several Azure offerings help you maintain and manage hybrid clouds such as Azure Stack, Hybrid SQL Server, and Azure StorSimple. Microsoft’s long history of working on enterprise IT gives them an upper hand when it comes to the hybrid cloud.

While Amazon realizes that it needs to strengthen its offerings to support hybrid clouds, it is still catching up, with more investments earmarked for hybrid clouds, according to Brian Olsavsky, Amazon’s chief financial officer. Still, the retail giant currently has a handful of solutions that is geared for companies who wants hybrid cloud deployments such as Storage Gateway, Direct Connect, and DynamoDB Local.

Azure vs AWS for Microsoft Shops

Microsoft has long been synonymous with larger enterprise customers. Microsoft Azure makes it easy for those currently using Windows Server, SQL Server, Exchange, and other Microsoft technologies to move to the cloud.

For .NET developers, publishing your application to Azure is amazingly simple. Publishing an app to Azure App Services or Cloud Services takes away all of the headaches of deploying apps and managing servers.

For Microsoft shops, Azure will hold a strong edge. Although, AWS supports Windows, SQL Server and other technologies that .NET developers are used. Amazon AWS has a great SDK for .NET. AWS might be more compelling for .NET developers if there is a particular AWS feature that is needed, that does not have an Azure equivalent.

Learn more about how Stackify uses Azure: Azure lets Stackify easily empower fellow developers

Azure vs AWS for Open Source Developers

Amazon might have started off as merely an online seller, but Microsoft has consistently had its eye on business customers focusing on Windows and similar platforms.  Azure continues this rapport with enterprise users by making sure that integration with Visual Studio is smooth, as well as the integration with Active Directory.  You can even use your current Active Directory account to sign on the Azure platform and Office 365.

However, Amazon shines when it comes to open source developers.  Microsoft has historically been very closed to open source applications, and it turned a lot of companies off.  AWS, on the other hand, welcomed Linux users and offered several integrations for open source apps.

In recent times, Microsoft has openly embraced open source technologies. Microsoft recently opens sourced the .NET Framework and the new .NET Core runs on Windows, MacOS, and Linux. SQL Server now runs on Linux. Microsoft also claims that about 1/3 of Azure Virtual Machines are running Linux and some of the infrastructure that drives Azure even uses Linux.

Government Cloud

When you think of the cloud, you probably think of startups and big tech companies. IT leaders working in the government are also working on moving their apps to the cloud. Government websites and other cloud deployments need to adhere to certain regulations, which is why putting government assets on public clouds raises warning flags as far as compliance is concerned.

The good news is that both Azure and AWS have a special section for government users.  These government clients make use of this isolated area so that their workloads do not share the computing, networking and other resources with ordinary business users.  Azure and AWS promise compliance with various regulations, such as the HIPAA, DISA, FIPS, among many others.

MORE: Azure Government Cloud & AWS Government Cloud

License, Fees and License Mobility

AWS has always made it a point to give customers headache-free licensing.  It is all a matter of paying for the licenses that you use no matter what AWS is offering you avail of.  But if you have Microsoft licenses that you’ve already paid for, you might be eligible for license mobility.  This means that you do not have pay double for using the same Microsoft server applications you’re currently using.

Azure also has the same license and mobility standards.

Do not assume, however, that all your Microsoft licenses are eligible.  For instance, Windows Server is not a part of the list of available applications, but Exchange Server, SQL Server, Skype, System Center Server and Project Server are.

Estimated costs for the use of either AWS or Azure might be a little difficult to come up with, but both services currently offer cost calculators that users can use to get an idea of how much they can expect to spend on each platform.

Features

On a per feature basis, you will find that most of all features offered on Azure have a corresponding or similar feature on AWS.  And while it will be quite difficult to come up with an exhaustive features list, you might find it interesting that some Azure services have no AWS equivalent. These include the Azure Visual Studio Online, Azure Site Recovery, Azure Event Hubs, and Azure Scheduler.  However, it seems that AWS is trying to close the gap.  For instance, AWS now offers AWS Lambda on preview to counter Azure’s Logic Apps.

In the end, choosing between Azure and AWS would depend on what you need and what they offer.   Which one is better? There is simply no blanket and definitive answer to that question.  Both AWS and Azure have free offerings and trials, so give each one a test run to help you get a feel of what to pick!

Cloud Services Comparisons

The battle between Microsoft Azure and Amazon Web Services has been heated for months, and experts everywhere are placing bets on who’s going to win the race. For more insights on the ongoing battle between these two leading cloud vendors, check out the following resources:

Azure vs. AWS vs. Google Compute

Compare Services & Features Between Microsoft Azure vs. Amazon Web Services vs. Google Compute

Angela Stringfellow Developer Tips, Tricks & Resources, Insights for Dev Managers, Live Queue Leave a Comment

The easiest way to compare the big cloud services players is by evaluating products, services, and features in a direct comparison to determine which cloud best meets your needs. So we’ve done just that, pitting Azure, AWS, and Google Cloud against each other in this head-to-head comparison chart. (For a detailed discussion comparing Azure and AWS, check out this post.)

Microsoft Azure Amazon Web Services (AWS) Google Compute
Available Regions Azure Regions AWS Global Infrastructure Google Compute Regions and Zones
Compute Services Virtual Machines (VMs) Elastic Compute Cloud (EC2) Compute Engine
App Hosting Cloud Services
Azure Websites and Apps
Azure Batch
Azure Scheduler
Logic Apps
Amazon Elastic Beanstalk Google App Engine
Serverless Computing Azure Functions AWS Lambda Google Cloud Functions
 ALM & Code Editor Azure Visual Studio Online AWS CodeDeploy None
Container Support Docker Virtual Machine Extension (how to) EC2 Container Service

Container Engine
Container Registry
Scaling Options Azure Autoscale (how to) Auto Scaling Autoscaler
Analytics/Hadoop Options HDInsight (Hadoop) Elastic MapReduce (EMR) Google Cloud Dataproc
Government Services Azure Government AWS GovCloud None
App/Desktop Services Azure RemoteApp Amazon WorkSpaces
Amazon AppStream
None
Object Storage Azure Storage (Blobs, Tables, Queues, Files) Amazon Simple Storage (S3) Cloud Storage
Block Storage Azure Blob Storage (how to) Amazon Elastic Block Storage (EBS) Persistent Disk
Hybrid Cloud Storage StorSimple AWS Storage Gateway None
Backup Options Azure Backup Amazon Glacier Google Cloud Storage
Disaster Recovery Planning Azure Site Recovery None None
Content Delivery Network (CDN ) Azure CDN Amazon CloudFront Cloud CDN
Database Options Azure SQL Database Amazon Relational Database Service (RDS)
Amazon Redshift
Cloud SQL
Cloud Spanner
NoSQL Database Options Azure DocumentDB Amazon Dynamo DB Cloud Bigtable
Cloud Datastore
Caching Azure Managed Cache (Redis Cache) Amazon Elastic Cache None
Data Orchestration Azure Data Factory AWS Data Pipeline BigQuery
Cloud Dataflow
Networking Options Azure Virtual Network Amazon VPC Cloud Virtual Network
Azure ExpressRoute AWS Direct Connect Cloud Interconnect
Azure Traffic Manager Amazon Route 53 Cloud DNS
Load Balancing Load Balancing for Azure (how to) Elastic Load Balancing Cloud Load Balancing
Administration & Security Azure Active Directory AWS Directory Service
AWS Identity and Access Management (IAM)
Cloud Identity & Access Management (IAM)
Multi-Factor Authentication Azure Multi-Factor Authentication AWS Multi-Factor Authentication Cloud Identity-Aware Proxy (IAP) (Beta)
Security Key Enforcement
Monitoring Azure Operational Insights Amazon CloudTrail Cloud Console
Azure Application Insights Amazon CloudWatch Stackdriver Monitoring
Stackdriver Logging
Queueing Azure Service Bus
Azure Event Hubs
Amazon Simple Queue Service (SQS) Cloud Pub/Sub
Notifications Azure Notification Hubs Amazon Simple Notification Service (SNS) None
Secure Credentials Azure Key Vault (Preview) AWS Key Management Service Cloud Key Management Service
Compliance Azure Trust Center AWS CloudHSM Google Cloud Platform Security
Management Services & Options Azure Resource Manager Amazon CloudFormation Cloud Deployment Manager
API Management Azure API Management Amazon API Gateway Cloud Endpoints
Automation Azure Automation AWS OpsWorks
AWS Config
Compute Engine Management with Puppet, Chef, Salt, and Ansible
Automated Image Builds with Jenkins, Packer, and Kubernetes
Search Service Azure Search Amazon CloudSearch None
Analytics Azure Stream Analytics Amazon Kinesis Cloud Dataflow
Cloud Dataprep (Beta)
Email Services None Amazon Simple Email Services (SES) None
Media Services Azure Media Services Amazon Elastic Transcoder
Cloud Video Intelligence API
Machine Learning Azure Machine Learning (Preview) Amazon Machine Learning Cloud Machine Learning Engine
Workflow Azure BizTalk Services Amazon Simple Workflow (SWF) None

So, which cloud is the right choice for you? Sometimes, it’s best to try before you buy, so narrow down your options based on the services and features you need, then take advantage of a trial period to find out how well the cloud service you’ve chosen will meet your real-world requirements.

ASP.NET Core Web Servers: Kestrel vs IIS Feature Comparison and Why You Need Both

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

The Kestrel web server is a new web server as part of ASP.NET Core. It is now the preferred web server for all new ASP.NET applications. In this article, we will review what it is, how to use it, and the differences between Kestrel vs IIS.

Why Do We Need the New Kestrel Web Server? What about IIS?

If you have been developing ASP.NET applications for a while, you are probably familiar with Internet Information Services (IIS). It does literally anything and everything as a web server. It is infinitely configurable with ASP.NET handlers & modules via the ASP.NET integrated pipeline. It has robust management APIs for configuration and deployment. It is even an FTP server.

The same codebase that has to support the original “.asp” pages from 15+ years ago now also handles new technologies like async ASP.NET. Like most software, as it ages it gets modified over time, they carry a lot of weight and bloat. IIS does everything, but it is not the fastest web server around. Lightweight web servers like Node.js and Netty make IIS look old and slow.

A Chance to Start Over

By creating the Kestrel web server, the .NET community was able to start over from scratch. They no longer had to worry about backward compatibility for technologies that were 15+ years old. They could take all of their past knowledge to build the simplest and fastest web server possible. That is exactly what they did. Kestrel and ASP.NET Core were built for speed.

Kestrel is more than just a new web server. ASP.NET Core & Kestrel combined are a whole new request pipeline for how ASP.NET requests work. Things like HTTP modules & handlers have been replaced with simple middleware. The entire System.Web namespace is gone. Another big advantage is designing a web server to take advantage of async from the ground up. Performance is now a feature of ASP.NET.

Built for Speed

One of the big problems with IIS and the existing ASP.NET pipeline was the performance of it. For most real world applications, the performance is perfectly fine. However, it lagged way behind in benchmarks. The combination of Kestrel & ASP.NET Core has been shown to be many times faster. It is great to see the team putting performance as a top priority.

Granted, benchmarking an ASP.NET request that says “hello world” is not comparable to most real applications that do multiple SQL queries, cache calls, and web service calls in a single request. ASP.NET makes it easy to do most I/O operations asynchronously. ASP.NET Core & Kestrel have been designed from the ground up to take advantage of async. Most real world apps should perform better if the developers follow good best practices around using async.

Cross Platform

If the goal was to get ASP.NET running on Linux, that meant porting IIS to Linux or making ASP.NET work without IIS. Kestrel solved this problem. As a developer, I can write my ASP.NET application and deploy it to Windows or Linux either one. Kestrel works as my web server on both. However, it is still recommended to use IIS, Apache, or NGINX as a reverse proxy in front of it. Next, we will discuss why that is.

Comparing Kestrel Web Server vs IIS

IIS does almost everything. Kestrel does as little as possible. Because of this, Kestrel is much faster but also lacks a lot of functionality. I would think of Kestrel as really more of an application server. It is recommended to use another web server in front of it for public web applications. Kestrel is designed to run ASP.NET as fast as possible. It relies on a full fledged web server to do deal with things like security, management, etc.

Microsoft suggests always using another web server in front of Kestrel for public websites.

 

Kestrel vs IIS Why Use Both

 

Feature Comparison for Kestrel vs IIS

Here is an IIS vs Kestrel comparison of some key features. This should help you better understand the limitations of Kestrel. You can overcome these limitations by pairing it up with IIS or NGINX.

IIS Kestrel
Platform Support Windows Windows/Linux/Mac
Static Files
HTTP Access Logs
Port Sharing / Multiple apps*
SSL Certificates Internal**
Windows Authentication
Management Console
Process Activation (start it up)
Application Initialization (warm it up)
Configuration API
Request Filtering & Limits
IP & Domain Restrictions
HTTP Redirect Rules
WebSocket Protocol Middleware
Response Output Caching
Compression Optional Optional
FTP Server

* Port Sharing is a big limitation! This means you can’t host “site1.stackify.com” and “site2.stackify.com” both on port 80 on the same server, running different ASP.NET applications.

** Only works with internal SSL traffic between the reverse proxy web server and ASP.NET.

To learn more about ASP.NET server hosting technology, check out the Microsoft docs: Web server implementations in ASP.NET Core

Need help monitoring your ASP.NET Core applications?

Stackify Retrace’s APM solution works perfectly with ASP.NET Core!

Software Error vs Exception – In Real World Examples

Matt Watson Developer Tips, Tricks & Resources, Live Queue Leave a Comment

After 15+ years of software development, I still use the words error and exception interchangeably.

But is there a difference between exceptions and errors?

I think it is best to make the distinction with some examples of errors vs exceptions.

Example #1: Dishwasher Errors

Let’s use this example of a dishwasher. Both images depict a problem, or error, that happened with the dishwasher. One was clearly user error of putting in the wrong type of soap. The other was an error that the dishwasher recognized and showed an error code for this.

I think this dishwasher example perfectly sums up the difference between an error and an exception. Both of these problems were errors. The difference here is the designer and engineers designed the product to handle one of the errors. It was an exception that the engineering team knew how to detect and handle. The engineering team did not detect or handle the soap problem.

Exceptions are a type of error that is expected or known to occur.

Example #2: Software Avoiding a Car Crash

Let’s take this example of a Tesla crash. The software within the car was able to detect a stopped vehicle. The software was designed to recognize this as a known problem, or as an exception to normal behavior. The car was not able to completely prevent the crash. But, the developers were able to execute special logic to slow the car down to at least minimum the effect of the problem. Exception handling logic is key to good software. As developers, we have to anticipate the types of errors that may occur and handle them properly.

tesla stopping

 

Example #3: ATM

When you go to an ATM to withdraw money, you expect to actually get money, right? What if you don’t have any money in your account? That would be a good example of an InvalidBankBalanceException.

If the developers didn’t handle this correctly, they could be handing out money to people who shouldn’t be receiving it.

134979

Or what if the ATM itself didn’t have any money? It would not be good if they deducted $100 from your account for the money you were withdrawing, but it didn’t actually give you any.

20151221_061614-1024x576

 

How to Anticipate Application Errors

One of the keys to good software is good error and exception handling. It is a good best practice to always be on the defense as your write code. You have to pretend that everything is going to fail.

If you don’t handle possible error scenarios, inevitably, they will happen and your users will have problems. Granted, some of them might be user errors and there may not be anything you can do about them. The dishwasher soap example is a good example of this. Of course, my favorite says is “if it can be null, it will be null”.

Always anticipate possible exceptions and handle them properly.

public static void DoIt(string url)
{
	try
	{
	   WebClient client = new WebClient();
	   string response = client.DownloadString(url);
	}
	catch (WebException ex) when (ex.Response != null)
	{
		var response = ex.Response as HttpWebResponse;
		Console.WriteLine("Error receiving web response. Status: " + response?.StatusCode);
	}
	catch (WebException ex) 
	{
		Console.WriteLine("Error connecting to remote server: " + ex.Message);
	}
	catch (Exception ex)
	{
		Console.WriteLine("Unknown error occurred: " + ex.Message);
	}
}

Be sure to check out our guide to exception handling.

Conclusion

I hope you enjoyed this real world guide to an application error vs exception. As a developer, part of our challenge is trying to anticipate all the problems that could happen. Sadly, we never can anticipate all of them. This is why all developers need a good error monitoring system.

soccer-save-gif-4

 

The Best Load Testing Tools

Top Load Testing Tools: 50 Useful Tools for Load Testing Websites, Apps, and More

Angela Stringfellow Developer Tips, Tricks & Resources, Live Queue Leave a Comment

As every developer knows, what works in the ideal scenario (a.k.a. your production environment) won’t necessarily work in the real world. That’s why you need tools like Retrace to help you quickly track down bugs and performance issues to keep things running smoothly.

Load testing, just one of many valuable testing methods you can use before release to ensure that your application is bug-free and runs seamlessly, is the process of testing whether an application can withstand the load of concurrent users, requests, and data volumes when in use. In other words, load testing is a must for determining if your applications can stand up to real-world use conditions.

There are many load testing tools available today for all kinds of applications and organization. In this article, we’ll look at 50 different load testing tools and highlight the key features of each of them. Note: The following 50 load testing tools are not ranked or rated in any particular order of importance, but are listed in random order below.

1.  Apache JMeter

 

@ApacheJMeter

Apache JMeter

Apache JMeter is a 100% pure Java, Open Source application that can be used to test performance on both static and dynamic resources, simulating heavy loads on individual servers, groups of servers, networks, or objects to test strength and analyze performance under different load types. Check out the getting started documentation here.

Key Features:

  • JMeter is a Java application for load and performance testing
  • Used to test web apps, SOAP & REST web services, FTP, databases, and more
  • The most mature, and widely used open source load testing tool available
  • Many commercial tools support JMeter

Cost: Open Source

2. WebLOAD from RadView

 

@RadViewSoftware

WebLoad from Radview

WebLOAD from RadView powers the most demanding performance professionals in the world with enterprise-scale load testing.

Key Features:

  • Lets you record actions and produces a test script in JavaScript
  • Visually intuitive interface makes it easy to create and manage load testing scripts
  • Enables performance testing for web, server-side, cloud, and mobile applications
  • Well integrated with enterprise products from Oracle, SAP, Microsoft, and more

Cost:

  • Free Edition available
  • Contact for a quote

3. LoadComplete from SmartBear

@SmartBear

LoadComplete from SmartBear

A desktop tool for load, stress, and scalability testing for both websites and web apps, LoadComplete by SmartBear doesn’t require advanced coding skills – meaning you can focus on the more important tasks. Check out LoadComplete’s documentation here.

Key Features:

  • Load testing for .NET and Java apps
  • Generates various levels of traffic without requiring code inputs
  • Test options available for IE, Chrome and Firefox
  • Provides checks for both, secure and non-secure traffic

Cost: Plans start at $230 – $1,625 per month, based on a 1-year subscription

4. NeoLoad

@Neotys

NeoLoad

A load and performance testing tool that realistically simulates user activity, NeoLoad helps you eliminate bottlenecks in your web and mobile apps. Check out the documentation here.

Key Features:

  • Supports Selenium script conversion for Selenium C# developers
  • Provides ‘drag and drop’ loops, conditions and other controls
  • Checks user paths to find newer path versions to avoid errors
  • Makes recording faster by providing prefilled transaction list

Cost: Contact for a quote

5. Locust

@locustio

Locust

Locust is an Open Source load testing tool that enables you to define user behavior using Python to test your system’s performance under millions of simulated, simultaneous users. Full Locust documentation is available here.

Key Features:

  • Allows you to write expressive scenarios in plain-old Python
  • Enables testing on any system, and multiple systems at the same time
  • Uses community members to add to the traffic

Cost: Free

6. BlazeMeter

@BlazeMeter

BlazeMeter

A performance engineering platform for DevOps, BlazeMeter enables you to run and analyze JMeter and other open-source load tests from anywhere. Check out the documentation here.

Key Features:

  • Quickly pinpoints defects using inbuilt error report function
  • Detailed and interactive report timeline graph to make the report clearer
  • Thorough testing using both, API backend and frontend
  • Uses YAML and JSON syntax for developers using these languages

Cost:

  • Free for 50 concurrent users, 10 tests, 1 shared load generator
  • Plans start at $99 – $499 per month

7. Load Multiplier

@exec_hr

Load Multiplier

 

Load Multiplier is a testing framework to test IT products across domains such as telecommunication, banking, web, proprietary protocols. Additionally given a client library, it can generate millions of such client instances so as to generate real field traffic. It comes with a web based front end, and powerful C/C++ based back end engine.

Key Features:

It provides stable uninterrupted long hour load testing solution for products based on:

1. WebRTC
2. IoT
3. SIP, IMS
4. RTP, RTCP
5. STUN, TURN, ICE
6. HTTP, SOAP, REST
7. JSON, XML
8. G711u, G711a, AMR, OPUS, VP8
9. Any proprietary text / binary protocol

Cost: Contact for a quote

8. WAPT

@onloadtesting

WAPT

A load and stress testing tool for easily analyzing website performance, WAPT creates tests in minutes for mobile applications, web services or customized ERP systems. The full WAPT documentation is available here.

Key Features:

  • Sends reports to mobile device and desktop device for convenience
  • Provides extensions to work with ASP.net, Adobe Flash and Silverlight
  • Test results are shown in 20 different detailed reports
  • Provides detailed graphs even during the runtime, to help track the performance

Cost:

  • WAPT: Starts at $700
  • WAPT Pro: Starts at $1,200

9.  Test Studio from Telerik

@Telerik

Test Studio from Telerik

A complete test automation solution for GUI, performance, load, and API testing, Test Studio from Telerik is a useful tool for testing desktop, web, and mobile applications. The full documentation is available here.

Key Features:

  • Provides testing for HTML5, AJAX, Android, WPF
  • Allows users to share and pick references from Element Explorer
  • Developers can use any language they are used to
  • Users can test any NativeScript app over 500+ devices

Cost:

  • DevCraft: $1,499/developer (perpetual license)
  • Telerik Platform: Starts at $39 per month per developer

10. Loader.io

@loaderio

Loader.io

Loader.io offers simple cloud-based load testing for stress testing your web apps and APIs with thousands of concurrent connections. Check out the documentation here.

Key Features:

  • Runs tests for you anytime and from anywhere
  • Allows monitoring of the test and performance from any place
  • Provides stress test result in descriptive graphs which will be sent to all users

Cost:

  • Free: 10k clients/test
  • Pro: $99.95 per month – 100,000 clients/test

11. HPE LoadRunner

@HPE_Loadrunner

HPE LoadRunner

Get a complete picture of end-to-end system performance to identify and resolve issues before your apps are released with HPE LoadRunner. Check out the LoadRunner documentation here.

Key Features:

  • Reduces the time it takes to understand and explain the reports
  • Supports wide range of applications to decrease time and skill
  • Provides detailed mobile performance test reports
  • Combines load testing with development tools like IDE, jUnit and more

Cost: Contact for a quote

12. StormRunner Load from HP

@HPE

StormRunner Load

An SaaS-delivered cloud load and performance testing solution, StormRunner Load makes it simple to plan, run, and scale testing for both web and mobile apps. You can read the full documentation here.

Key Features:

  • Takes 10 minutes to design and create web load or mobile test
  • Tests can be managed from anywhere using a cloud based testing function
  • Can quickly scale from 1 to 1,000,000 geographically distributed web and mobile user

Cost: Contact for a quote

13. CloudTest from Soasta

@SOASTAInc

CloudTest from Soasta

CloudTest provides scalable and affordable load testing tools for more efficient, transparent performance testing. Check out the full CloudTest documentation here.

Key Features:

  • Gets real user experience from 95 different locations in all 7 continents
  • Provides you with 40 cloud service providers to make your testing seamless
  • CloudTest automation tools quickly and easily integrate with yours
  • Importing existing JMeter test scripts have been made easy

Cost:

  • Lite: Free for 100 virtual users
  • On-Demand: Starts at $2,500

14. The Grinder

@philip_aston

The Grinder

A Java load testing framework, The Grinder makes running distributed tests simple using many load injector machines. The Grinder is maintained by Philip Aston, who took over the code (originally developed for the book Professional Java 2 Enterprise Edition with BEA WebLogic Server by Paco Gómez and Peter Zadrozny) and reworked it to create The Grinder 2 and subsequently The Grinder 3. The full documentation can be found here.

Key Features:

  • Test scripts can be written in Clojure and Jython
  • Any Java API can be loaded as a test, be it from HTTP web servers, SOAP and REST services, etc
  • Pre-built Java libraries with large variety of systems and protocols is available

Cost: Free

15. Tsung

Tsung

An Open Source, multi-protocol, distributed load testing tool, Tsung helps you test the performance and scalability of IP-based client/server applications. Check out the full documentation here.

Key Features:

  • Multiple IP addresses can be opened on one machine using their OS IP Aliasing
  • Response time can be measured during the load by generating HTML reports
  • HTTP, WebDAV, MySQL and LDAP servers can stressed using this software

Cost: Free

16. Gatling.io

@GatlingTool

Gatling

An Open Source load and performance testing tool for web applications, Gatling helps you improve time-to-market by identifying bottlenecks, errors, and performance issues prior to release. Gatling documentation is available here.

Key Features:

  • Can forecast performance issues in the web application you make
  • Errors and bottlenecks will be detected in the early part of the development cycle
  • Drastically reduces the debugging phase time
  • Provides the best digital experience to customers and users

Cost: Free

17. Taurus

Taurus

Because automating repetitive tasks just makes sense, Taurus provides an automation-friendly framework for continuous testing. You can read the full documentation here.

Key Features:

  • Provides smooth automation and integration of JMeter and other systems
  • Creating, running and analyzing performance tests are made simple
  • Has a different approach and system than The Grinder and Gatling
  • New tests can be made from scratch using control friendly and unified DSL

Cost: Free

18. Httperf

Httperf

Httperf is an Open Source HTTP load generator for measuring web server performance that facilitates the construction of both macro- and micro-level benchmarks.

Key Features:

  • This is a simple open source tool for measuring web server performance
  • It is solely made to test standard http payload of the application you test
  • High performance and robust tool that helps create micro and macro benchmarks
  • It is able to create and sustain server overloads

Cost: Free

19. Silk Performer from Micro Focus

@MicroFocus

SilkPerformer

Micro Focus Silk Performer understands that the biggest and most complicated issues happen at the most inconvenient times, so Silk Performer is designed to provide a consistent user experience anywhere, anytime, on any device. Check out the documentation here.

Key Features:

  • Load tests reflect real world user behavior with their different constraints
  • Provides unlimited cloud support for load testing on mobile devices and systems
  • Prepares application for low bandwidth users

Cost: Contact for a quote

20. ApacheBench

@TheASF

ApacheBench

A tool designed for benchmarking an Apache Hypertext Transfer Protocol (HTTP) server, ApacheBench gives you an accurate picture of how your current Apache installation performs. The documentation is available here.

Key Features:

  • Provides better support for asynchronous read and write
  • Keepalivetimeout can be specified in a millisecond
  • Helps you understand how your current Apache installation performs

Cost: Free

21. ZebraTester

@apicasystems

ZebraTester

Offering powerful tools and flexible options, ZebraTester enables you to easily create and run tests of nearly unlimited scale with incredible visibility into your results. Check out ZebraTester’s documentation here.

Key Features:

  • Includes performance testing for CI/CD process using LoadTest Portal
  • Apica team helps design, execute and manage the project
  • Security policies have been given equal importance, and subsidiary software is available
  • Free 500VU are provided on their global network and LoadTest Portal

Cost: Contact for a quote

22. Experitest Mobile Add-On for LoadRunner

@Experitest

Experitest

Experitest offers several testing tools for mobile apps, including a mobile add-on for HP LoadRunner, which makes it possible to perform mobile load testing by connecting real devices and testing your applications under simulated loads. Read the full documentation here.

Key Features:

  • This is specially made to test your mobile applications load taking capacity
  • Runs the same tests as systems on the mobile devices
  • Create and edit mobile tests inside LoadRunner
  • Creates report screenshots and videos of the application

Cost: Contact for a quote

23. Load Impact

@LoadImpact

Load Impact

With nothing to install and easy configuration, Load Impact offers instant load testing for devops and developers. Read the full documentation here.

Key Features:

  • Connects you with performance trending analytics and notifies through webhook
  • Provides a powerful Lua scripting environment to allow you to create simple or complex API scenarios
  • Record results of test uploaded on website or app using their chrome extension
  • They have provided more than two million customers with load testing service

Cost: $89 – $1,299 per month

24. AgileLoad

@AgileLoad

AgileLoad

Optimize your load and performance testing costs with AgileLoad, which provides distributed and cloud-based testing on-demand and server and database monitoring in a single package. Documentation for AgileLoad can be found here.

Key Features:

  • Helps you create complicated scenarios in minutes using their Web 2.0 technology
  • Allows you to simulate your load from any location, through the Cloud
  • Gives you an accurate end to end picture of your application’s performance
  • Lets you correlate different load test by combining them in one graph

Cost:

  • Starts at $192 per month
  • Daily rental starting at $24 per day

25. LoadStorm

@LoadStorm

LoadStorm

An easy and cost-effective way to load test both web and mobile apps, LoadStorm helps you find the breaking point of your applications and other weaknesses before your customers do. Check out LoadStorm’s Getting Started documentation here.

Key Features:

  • Provide full project management, where they do all the test work for you
  • Cloud servers are able to run load tests up to 1,000,000 VUsers
  • Allows logins from any location around the world
  • Provides control of spreading the traffic to any geographic location

Cost: Contact for a quote

26. Visual Studio Team Services

@VSTeam

Visual Studio Team Services

Visual Studio Team Services offers cloud-based load testing to performance test your code under the weight of thousands of users. Check out the documentation here.

Key Features:

  • Uses Azure to make Cloud-based load testing easy and quick
  • Visual Studio Team Service can help make tests by your references of other websites
  • Correlates test results with server diagnostics
  • Functions to record and replay your actions are also available

Cost:

  • Free up to 5 users
  • Growing Teams: Starts at $30/month for 10 users

27. Loadster

@loadsterperf

Loadster

A full-featured testing solution for websites, web apps, and web services, Loadster is built for real web applications and can handle cookies, user sessions, custom headers, and more with ease. Check out Loadster’s documentation here.

Key Features:

  • Gives a hybrid solution for load testing with ground server and cloud system
  • Provides two options for generating concurrent user, from their server and your own
  • Concurrent users are generated from 5 different continents
  • Loads multiple tests at once and generates users for all tests

Cost: $79 – $2,999

28. LoadView

@dotcom_monitor

LoadView

LoadView provides flexible load testing from the cloud for websites, web apps, mobile, and APIs. With a 100% managed cloud, there’s no need to set up your own third-party cloud accounts.

Key Features:

  • Complete Cloud-based load testing software, eliminating hardware hassle
  • Allows you to pay according to the use if you don’t want to pay for a month
  • Shows you the load stress for real world browsers
  • Trusted by Volvo, Dell and ComCast

Cost (per load test):

  • Each Virtual Machine: $4
  • HTTP Load: $0.01 per user per minute
  • Full Browser Load: $0.10 per user per minute
  • Flat monthly cost to store test result data: $9.99

29. Artillery

@ShoreditchOps

Artillery

A modern, powerful, and easy-to-use load testing toolkit, Artillery empowers developers to make scalable, performant, and resilient applications that can stand up to high load. The documentation is available here.

Key Features:

  • Supports multiple protocols- HTTP(S), WEbSockets and Socket.io
  • Software prefered by ecommerce backends, loT bankends and transactional APIs
  • Supports Statsd, Datadog, InfluxDB and Librato
  • Use npm modules and customize load tests with Java Script

Cost: Free

30. eggPlant

@testplant

eggPlant

Ensure that your cloud and server-based applications will continue to work, perform flawlessly, and provide a consistent user experience with eggPlant. Check out eggPlant’s documentation here.

Key Features:

  • Technology support includes client-side .NET, client-side Java, Citrix and TCP/UDP
  • Load test creating tools are easy, which helps new and experienced users design
  • C# and Java are fully integrated to help faster script authoring
  • Provides live monitoring to help control tests while they run

Cost: Contact for a quote

31. Applause

@applause

Applause

Offering a range of load testing services, Applause helps you ensure that your apps and websites can handle the biggest loads on your most successful days. Check out the documentation here.

Key Features:

  • Pre-built custom load scripts to reduce scripting time and money
  • Provide expert help to interpret load test results for better insights
  • Give access to experts who craft custom Apache JMeter load testing scripts
  • Provides cloud-based infrastructure for flexible scaling

Cost: Contact for a quote

32. LoadUI

@soapui

LoadUI

LoadUI provides load testing for REST and SOAP APIs to ensure your APIs never let you down. You can rapidly create API load tests either against a single web service endpoint or an existing functional API test.

Key Features:

  • Allows reuse of existing SoapUI Pro functional tests
  • Can run several load test scenarios continuously with one another
  • See the interaction of each scenario and the impact they have
  • Shows how your server reacts to traffic and diagnose the bottlenecks

Cost: $4,999 – $14,999 per year

33. JCrawler

JCrawler

JCrawler is an Open Source stress testing tool for web apps with a crawling/exploratory feature enabling you to define a set of starting URLs for JCrawler to begin crawling and generating loads. Check out the documentation here.

Key Features:

  • Can be used through little bandwidth, doesn’t require high internet speeds
  • Uses XML file to configure, which makes it easier to create load tests
  • Can be run on any type of system
  • JCrawler tools specially made for developers and QAs

Cost: Free

34. StresStimulus

@StresStimulus

StresStimulus

A load testing tool for both web and mobile applications, StresStimulus measures web performance and scalability under heavy traffic loads. The full StresStimulus documentation can be found here.

Key Features:

  • Records traffic for mobile devices like Apple, Android, Windows and Blackberry
  • Works smoothly with all main web platforms- Java, PHP and ASP.NET
  • Supports many Enterprise applications CRM, SharePoint, Silverlight and more
  • The only software that can be used as an add-on for

Cost: $1,040 – $3,950 per month (Perpetual 250 VUs License)

35. OpenSTA

OpenSTA

A distributed software testing architecture designed around CORBA, OpenSTA performs scripted HTTP and HTTPS heavy load tests with performance measurements from Win32 platforms. The documentation can be found here.

Key Features:

  • Made for experienced and proficient testing users
  • Records can be played by different users’ perspectives for better understanding
  • Open Source tools licensed under GNU General Public License
  • Was made to be free for user and will always remain free

Cost: Free

36. SOAtest from Parasoft

@Parasoft

SOAtest

With SOAtest from Parasoft, you can automate complete end-to-end testing for both business- and security-critical transactions. Check out the documentation here.

Key Features:

  • Provides a user-friendly interface for monitoring live application of traffic
  • The tests are engineered to shared, reused and extension easily
  • Includes static analysis, code review, code coverage analysis and runtime error detection
  • Designed to make load testing without scripting possible

Cost: Contact for a quote

37. APImetrics.io

@APImetricStats

APImetrics

You can’t afford latency in your APIs, and APImetrics aims to solve that challenge by putting real-time API performance data at your fingertips. APImetrics documentation can be found here.

Key Features:

  • Provides real-time alerts and notifications covering failures, speed and size issues
  • Daily, weekly, monthly and yearly review given through detailed reports
  • Gives you functionality for every sector you service
  • Allows you to measure the latency and uptime from any location or cloud

Cost: Starts at $10,000 per year

38. Goad

GOAD

An AWS Lambda powered, distributed load testing tool, GOAD enables you to launch HTTP loads from up to four AWS regions simultaneously. Check out GOAD’s documentation here.

Key Features:

  • Allows HTTP load launch from four different AWS at the same time
  • A single lambda is able to handle hundreds of concurrent connections
  • Peak load of up to 100,000 concurrent requests

Cost: Free

39. Reflective

Reflective

A library injection technique that employs the concept of reflective programming, Reflective DLL injection performs the loading of a library from memory into a host process. Check out the documentation on GitHub.

Key Features:

  • This follows the concept where reflective programming is employed
  • The library loads itself by using minimal portable execution
  • It interacts with the minimal interaction with the host system
  • Injection works on Windows NT$ and up including Windows 8

Cost: Free

40. LoadTracer

@Loadtracer

LoadTracer

A user-friendly tool for web application load testing and performance testing, LoadTracer works like many other load testing tools, recording scripts and playback with hundreds of virtual users.

Key Features:

  • It helps with web application load testing, monitoring and performance testing
  • Uses various techniques to record and find bottlenecks and latency issues
  • Also provides tools for website loading tests

Cost: Contact for a quote

41. Solex

Solex

A free, Open Source web application tool, Solex was built as a plug-in for Eclipse IDE. The full documentation can be found at SourceForge.

Key Features:

  • Provides functions to record client sessions and replay it later after adjusting
  • It acts as a HTTP proxy and records HTTP requests between Web client and server
  • It is an international Open source project, which makes it more helpful
  • It is based on Eclipse platform, making tools fully integrated

Cost: Free

42. Bees with Machine Guns

Bees with Machine Guns

Bees with Machine Guns is a nifty little utility that sends swarms of “bees” to attack (a.k.a. load test) your web applications. Check out the documentation here.

Key Features:

  • Helps create micro EC2 instances called bees to load test web applications
  • Uses Python 2.6 – 3.6, Boto, Paramiko dependencies
  • Can be pointed at any web application to test the load taking capacities

Cost: Free

43. Appvance

@appvance

Appvance

The first unified test automation platform that completely transforms the software QA process, Appvance eliminates silos and helps you see your app’s performance through your users’ eyes.

Key Features:

  • Tests your applications and shows how a user would see it
  • Finds bugs the user would find on their browsers and mobile applications
  • Data-drive is available for any language you use
  • Brings in a new level of smoothness in their record and play script creation

Cost: Starts at $99 – $1,200 per month

44. nGrinder

nGrinder

nGrinder is an enterprise-level performance testing solution based on The Grinder, allowing you to execute script creation, test execution, monitoring, and the result report generator simultaneously. Check out the documentation here.

Key Features:

  • Uses Jython and Groovy to design scenarios and create stress in JVM
  • Tests can be extended to limitless customs, using their unlimited library
  • Library consists Jar, Python, Maven dependencies
  • Allows you to monitor the agents creating stress and the targeted machines

Cost: Free

45. Rational Performance Tester from IBM

@IBM

Rational Performance Tester

Test earlier and more often with Rational Performance Tester, which validates web and web server applications for scalability, identifies bottlenecks, and more. Check out the documentation here.

Key Features:

  • Allows early and frequent tests of DevOps approach
  • It helps see the extent of the web and server scalability
  • Identifies the presence and cause of the performance bottlenecks and system issues
  • Provides root cause analysis feature to find and diagnose the issue

Cost: Contact for a quote

46. J-hawk

J-hawk

An Open Source Java-based framework that you can incorporate into your applications for performance testing, J-hawk generates performance reports to help you identify bottlenecks. Check out the documentation here.

Key Features:

  • An Open Source Java based framework to test the performance of the application
  • Uses its own scripting language called $hawk scripting
  • Provides two ways to use j-Hawk- Property Mode, Script Mode
  • j-Hawks solves what junits does not solve

Cost: Free

47. Selenium

@SeleniumHQ

Selenium

Selenium isn’t actually a load testing tool itself, although its powerful browser automation features enable various load testing scenarios for low-scale browser loads. Check out Selenium’s documentation here.

Key Features:

  • Though not a load testing tool itself, its powerful browser automation features enable various load testing scenarios
  • Provides two types of applications for user, based on the kind of work they want to execute
  • Selenium WebDriver creates robust browser tests and automation suites
  • Selenium IDE designs scripts to help Automation exploratory test

Cost: Free

48. Performance Center from HP

@HPE

Performance Center from HP

With Performance Center from HP, you can plan and execute tests across multiple global projects. With software testing tools and consistent processes, you can create a world-class testing center with ease. The documentation is available here.

Key Features:

  • Is updated to be compatible to all sorts of hardware
  • Creates a common testing infrastructure for engineers located around the globe
  • Use tests stored in the Cloud or in your premise behind a firewall
  • Reduces cost by creating a shared tests report infrastructure for engineers

Cost: Contact for a quote

49. Hammerhead

Hammerhead

A web site coverage, HTTP load generator, HTTP benchmarking, and stress testing tool, Hammerhead is designed to emulate several users from different IP addresses at maximum speeds to push your apps to the limit.

Key Features:

  • It is a HTTP load generator and benchmarking tools
  • Provides web site coverage and website stress testing tools

Cost: Free

50. Load Xen

@TestCollab

Load Xen

Load Xen, by Testcollab, enables you to simulate millions of requests to help you more effectively scale your applications. You can create interactions that seem just like real users to your application, randomize requests, run concurrent tests, and more with Load Xen.

Key Features:

  • Simulate hundreds to millions of virtual users
  • Create interactions that mimic real use
  • Flexible scripting
  • Real-time reporting
  • Concurrent tests

Cost:

  • Free: 100 VUs, max test time 2 hours
  • Paid plans: $50 to $500 per month

It is important to test any software’s performance and load handling capabilities before it is released. There are many options available ranging from open source to enterprise-ready tools to perform load testing. We hope this list will help you choose the right load testing tool for your project depending on the kind of software you are building and your budget.