The first thing every developer wants to see when their application isn’t working correctly is their logs. The only thing worse than not having any logging is not being able to access it quickly.
How many times have you begged a system administrator or another developer to send you the log files? Talk about a huge waste of time!
Log files are great but they don’t scale past your workstation… what you really need is a cloud logging service.
Applications are typically saved in files on disk. This works great on your local development machine.
However, it is terrible when you have an application that runs on multiple servers and has a lot of logging.
Common problems with log files:
Cloud logging services solve these problems and provide many additional features. Logging files tend to be a black hole and they rarely get looked at. A logging service can solve the problems listed above and make your logging data much more powerful.
Sending your existing logs to a centralized logging service is usually easy to do.
How you do you logging will vary greatly depending on your programming language of choice. If you are using common logging frameworks, it is usually very easy to send your logs to a new destination, like a logging service.
Here are some of the common logging frameworks:
If you are using any of these, the process is easy. If you are using one of these programming languages, but not using common logging frameworks, I would highly suggest that you do. They make it easy to “plug and play” send your logs to a wide array of places via configuration.
If your logs are stuck in a text file and you have no other options, you could look at using a tool like NXLog to parse them and then send them to a logging service.
As a simple example, here is how easy it is to configure NLog to send to my logs to Stackify Retrace.
I’m already using NLog and my logging configuration is in a file called NLog.config.
To send my logs to Stackify, I first add a NuGet package to my project.
PM> Install-Package NLog.Targets.Stackify
This downloaded Stackify’s libraries and added them to my project as references. Now all I need to do is modify my NLog.config file and add a couple application configuration settings to specify my Stackify license key.
Here is my new NLog.config file:
<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true">
<targets async="true">
<target name="logfile" xsi:type="File" fileName="file.txt" />
<target name="console" xsi:type="Console" detectConsoleAvailable="false" />
<target name="stackify" type="StackifyTarget" />
</targets>
<rules>
<logger name="*" minlevel="Trace" writeTo="logfile" />
<logger name="*" minlevel="Info" writeTo="console" />
<logger name="*" minlevel="Info" writeTo="stackify" />
</rules>
</nlog>
As you can see, send my logs to cloud logging service like Stackify Retrace is very simple.
Earlier I discussed some of the reasons that log files suck. They end up being a black hole for your critical logging data. By streaming your logs to a centralized cloud logging service, you can solve those problems and do some pretty cool things with your logs.
Let’s take a look at some of the things that most log management tools will do for you.
Searching log files is pretty rudimentary. The data is not structured and a lot of additional metadata that could be really useful is usually not logged to a file.
When your logs are sent to a logging service, they typically include a lot of additional fields that provide context.
Example fields:
Being able to search by these specific fields makes it much easier to find the information you are looking for.
For example, I can do a search for “level:warn” and quickly see only the log messages that were logging level warning.
While looking at these logs I was curious what other logging may have occurred. Since it relates all log messages to a transaction, I was able to grab that transaction ID and search by it.
Now I can see 2 related log messages.
One of the most powerful things you can do is log additional data or objects from your code. You can then search by those custom fields.
For example, all of our application we always log our client ID with our errors and logs. This makes it easy to look for issues related to a specific client.
Read More: What is structured logging and why developers need it
Now that you have all of your logs in a single place and have powerful searching capabilities, you can monitor your logs!
Your logs can a wealth of information that can be used for monitoring for specific application behaviors.
For example, once a day we have a process that does some billing. We setup some special log monitoring that looks for errors specific to our billing code. If it finds anything that matches out search, we send out an alert to our team. The last thing we want is issues billing our customers.
The errors that happen in our code are important to track. They are often our first line of defense when it comes to identifying application problems. Be it database timeouts, pesky null reference exceptions, or any other type of error.
Some cloud logging services can also provide additional reporting and alerts around your application errors.
Being able to track unique errors is really useful. Knowing when they first occurred, how often they occur, etc. There is no way to do that just by looking at text files.
In this article, we discussed why logging files to text files don’t work very well once you deploy your application to production. Cloud logging services, like Stackify Retrace, can help you view all of your logging data from a centralized location. We also covered some of the cool features that logging services can provide.
There are several log management solutions that are cloud based. They are all fairly similar and easy to use. Our product, Retrace, is very different. Retrace allows developers to do complete application performance management. Including code level performance tracing, application logs, errors, metrics, and much more.
Stackify's APM tools are used by thousands of .NET, Java, PHP, Node.js, Python, & Ruby developers all over the world.
Explore Retrace's product features to learn more.
If you would like to be a guest contributor to the Stackify blog please reach out to [email protected]
