tail logs across servers

How to Tail Logs Across Multiple Servers

Matt Watson Developer Tips, Tricks & Resources Leave a Comment

Application and server logs are the eyes and ears for most developers. Anytime there is a problem with software, the first thing developers want to see is the logs. Developers typically need access to multiple types of logs. Including logs directly from their application, but also logs from servers, web server access logs and more. Piecing this puzzle together across multiple servers can be like finding a needle in a haystack.

What is Log Tailing?

Tailing logs is in reference to watching them in near real-time as they occur. There are a lot of different ways to tail log files from text files. We have a whole blog post dedicated to that topic even. It can be done via command line or with popular text editors like Notepad++. This is useful if you are testing an application on your development workstation and want to watch your logs as they happen in real time.

Standard log tailing techniques work great for monitoring log files on a local workstation or server. However, if you want to tail log files across multiple applications or servers at once, a log management system may be your best solution. This also gives you the ability to do robust searching and other advanced features.

How to Tail Logs with Retrace Log Management

One of the awesome features of Retrace is the ability to tail your logs. You can view your logs in near real time across multiple servers or applications. Retrace provides indexing of your logs, and you can search by any field.

By the way, Retrace works with Syslog, Windows Events, common web server access logs and common logging frameworks for popular programming languages.

Log Tailing Across Multiple Servers via Retrace

Log Tailing Across Multiple Servers via Retrace

3 Common Use Cases Log Tailing Multiple Servers

Tailing logs can be useful for a wide variety of use cases. Let’s explore a couple of common use cases of why you might want tail your logs.

Use Case #1: Watching for Application Errors

One of the great features of a log management system like Retrace is the ability to do advanced searches across your logging data.

Let’s pretend you are trying to track down some unhandled exceptions in production.


Free Download

You could do a simple search based on the logging level and the name of your application. (You could also use Retrace’s built in quick filters to pick this criteria from a list.)

“level:error appname:stackifyapi”

Tail Application Logs Across Servers via Retrace

Tail Application Logs Across Servers via Retrace

We could now watch for any errors in near real-time that are happening within this application. This is a great use for log tailing multiple servers. We could see them across all servers or environments. You could also filter it down further.

Tailing for errors is a great idea to do right after a deployment. It almost never fails that a new release causes some new error. With log tailing, you can see it immediately when it happens!

Use Case #2: Reproducing a Bug in Production

Debugging in production is a lot easier if you can tail your production log files. Reproducing bugs in production is usually a nightmare because you can’t easily find the correct logs across multiple servers and potentially gigabytes of log data. Log searching and tailing help solve this problem!

If your logs contain a customer or user identifier that can search for, you can easily execute that search and watch for the logs as they happen. I would highly recommend using structured logging to log these types of details with all of your logging. It makes your logging data so much more powerful.

For example, at Stackify we always log a clientid field that represents our customer’s account number. That makes it easy to search our logs for issues that could be related to a specific client.

“clientidNumber:51233”

Use Case #3: Watching Web Server Access Log Activity

Concerned about some weird activity on your web server? You could search by IP address or other details in real time and watch your web server logs.

Get a list of the top IP addresses accessing your web servers:

Tail Logs View Top IP Addresses

Top IP Addresses via Retrace

You could then select any of these and watch the activity from them in near real time.

Tail Web Logs Multiple Servers via Retrace

Tail Web Logs Multiple Servers via Retrace

Free Tool: Use Prefix to View Request Traces and Logs on Your Workstation

If you are running a .NET or Java application on your workstation, you could also use Prefix to view what your code is doing, including your logs. Prefix, like Retrace, tracks the performance of your code and provides an incredible trace of what your code is doing. Included in this is your application logging. It is pretty amazing to see your SQL queries, HTTP web service calls, and application logging in context together.

Prefix provides this for free on your workstation. You can accomplish this on your servers with Retrace.

Summary

Log tailing is something all developers should be familiar with. It is very useful during development via command line, a text editor, or a free tool like Prefix. Once your application gets deployed to servers, a log management tool like Retrace makes it easy to search and tail across all of your logs. Stackify can help your development team from development to production.

Retrace starts at just $10 a month. You can start a free trial right now!

About Matt Watson

Matt is the Founder & CEO of Stackify. He has been a developer/hacker for over 15 years and loves solving hard problems with code. While working in IT management he realized how much of his time was wasted trying to put out production fires without the right tools. He founded Stackify in 2012 to create an easy to use set of tools for developers.