Automated Exception Handling, New Error Detection and Bugs.

Exception handling, new error, trends, and bugs detection, out of your log data, had become an almost mission impossible in a modern IT environment where applications are installed in multiple clouds.

Understand the correlation between all these pieces of information is a tough job without AI on your side.

** learn about XpoLog end to end solution for IT, Security, DevOps, Auditors, and Hybrid Cloud.

XpoLog’s Analytic Search has been around for quite a while, but our latest version brings many new gadgets and such a friendly user interface that makes exception handling, error detection, etc’, so much easier.

XpoLog’s unique log search engine not only finds what you are looking for, but also all the things you never even thought to look for in the first place.

The augmented log search comes with a complete log analysis and dishes it all out to you on a silver platter.

How cool is that? Pretty damn cool!

In this series of posts I am covering some of the ways you can benefit from XpoLog’s new features and enhancements.

This article will focus on how to get the most valuable information from your log4j event logs.

Please note: XpoLog features a huge analytic apps marketplace, built-in connectors of dozens of systems. You can easily handle exceptions, discover errors and problems and extract insights from any type of log data.

Automated Log analysis and problem discovery

before we demonstrate how you can use the augmented log search in finding exceptions, errors, trends, anomalies throughout your entire deployment, we will give you a quick view of our automated log analytics.

A quick view of some of our apps:

get log4j insights in just a few clicks from download

Each app contains ready to use dashboards and real-time reports.

Install the log4j app and get automated insights about errors and exceptions in your log data

When you download XpoLog, you will stream events in just a few clicks (quick deployment wizard) and right after you will be suggested with the relevant app for the type of logs, to get insights in just a few clicks from the download.

Have we mentioned that XpoLog7 is a great tool to have on your side? 🙂

Log search with automated exception handling and error discovery

By running Analytic Search on your log4j data, you can measure:

  • Your application performance and thread activity.
  • Create your own Apps for better monitoring.
  • Measure code activity with class and method analytics on log4j.
  • Build security analysis.
  • And create dashboards, charts, slide-shows, and make use of other visualization gadgets for maximum analysis.

Let’s dive into XpoLog log analyzer, and learn how you can auto-detect new errors, exceptions and bugs in log4j logs, and discover unknown messages.

From Search to Analytics

In an old post, I discussed simple and complex log search using XpoLog’s powerful search engine.

Already at this stage, XpoLog suggests analytic insights you may be interested in investigating further.

To put it simply, while XpoLog’s Search Engine gives you everything you asked for, XpoLog Analytics gives you everything else.

If you are searching for a string, a thread, an error, etc. in one or any number of logs, folders, applications, or servers, within a given time frame, XpoLog’s search engine will find and display all cases/events of the search request within all logs.

But XpoLog will also open the door for any other abnormality that may occur in these logs within this time frame, and this brings us to Analytics.

In other words, as soon as you conduct a search, XpoLog will already automatically present you with all other issues that you did not even know existed, be it errors, exceptions, unknown messages, or any other anomaly.

As an example, look at the simple search we did (in my previous post) looking for all log4j logs where the priority was ERROR.

Quick recap: Inside XpoLog Center, on the Search page, in the search field, we typed: priority=error in log.log4j server* The result looked like this: Log4j Post 5 taken from post 4 Below the graph, XpoLog displays all the events where the priority=ERROR.

But in the sidebar (see red rectangle), XpoLog has already suggested Analytic Insights, such as ERROR, java.io.EOFException, … and the list goes on.

These Insights may not necessarily appear in any of the events where priority=ERROR, but they do appear somewhere in these logs, and hence, they may indicate that something went wrong somewhere.

So already, at this stage when all you are doing is searching for something, XpoLog is already several steps ahead, analyzing, and inviting you to dig deeper to find the root of the issue.

From the Analytics Insight list, we can select one or more insights and either add them to the search, or use them to replace the search. We can then investigate the matter further, in XpoLog Analytics.

Inside XpoLog Log Analytics

The screen capture below shows the Analytics page.

The top section has a graph showing you the data distribution and the maximum severity of the events over the selected time-span.

Below the graph is a table showing the logs and folders in which these events were found.

Below this section is another table showing the 10 most severe errors that were found in any of these logs and folders. Analytics - apptag view When listing the logs, XpoLog lists the logs containing the problems with the high severity first (red), then all those with medium priority problems (orange), and lastly, those containing low priority problems (green).

XpoLog decides the severity level according to the highest severity anomaly found in the event.

You may be searching for an anomaly with a medium severity, but if, in the same event, another anomaly with high severity is found, the event as a whole will be marked as high priority. search failed to initialize hudson small In the screen capture above, a search was conducted for Failed to initialize hudson, which has medium priority, but within the same event, XpoLog found a hidden message, java.lang.OutOfMemoryError, which has high priority, thus bringing the entire event up to high priority.

Drilling down

From the initial Analytics page, which by default shows the total summary of all anomalies in all logs of your search, you can drill down for more specific details. For the sake of our example, let’s drill down into log4j: Analytics - apptag log4j drilldown with magnify The Analytics page has now drilled down to the log4j level (see screen capture below).

You can see the number of anomalies has been reduced, as has the amount of data being depicted.

The first table below the graph now contains only folders of the log4j applications and the second table shows the most severe log4j problems found. Analytics - apptag log4j after drilldown

Log4j Use Cases

Let’s have a look at a potential use case. Inside log4j is a tomcat folder. A user is complaining that tomcat will not start.

We don’t know what the problem is, so the easiest way to find out is to do a general search for anything abnormal going on in tomcat in the given time frame when the user was unable to start it.

Inside XpoLog Search, in the search field, we type the following query: * IN folder tomcat 8 In addition to the requested search results, XpoLog suggests many more analytic insights.

There could be many reasons why tomcat did not start, so from the Analytics Insight list, we will select java.net.BindException and replaced the * search with this query: “java.net.BindException” IN folder tomcat 8  The screen capture below shows how to replace the existing search query: right-click on the insight and then select Replace search: Search all - replace bind exception The search result for “java.net.BindException” IN folder tomcat 8 will look as follows: Search bind exception We now have completely new events in the list and we see that for the insight “java.net.BindException” that the “address is already in use”.

This is why the user was unable to start tomcat.

*** In another example, we created an App called hudson. Hudson worked for a while, but at some point it stopped.

We did not know why, so we did a search for hudson. In the search field, we typed: hudson IN folder tomcat 8 We got the following results: hudson with analytics div fixed Here we can see in the Analytics Insight list that there is a high priority error: Java.lang.OutOf MemoryError.

In this particular example, this error occurs in the first event on our list of events containing “hudson”.

We now already know why hudson stopped working. By zooming in and hovering over the graph, you also get a presentation of the high, middle, and low priority errors per moment in time.

Bottom line…

So we can see from these examples and use cases that while users are simply searching for known anomalies, or even searching for any anomaly without pre-knowledge, XpoLog is already analyzing everything else about the logs in the search that the user never thought of.

In the fraction of a second, it takes XpoLog to:

  • Do the requested search.
  • And also create a complete list of analytic insights, for the users that invites them to dig deeper into their logs, folders, apps or servers, to get to the root of whatever is causing them trouble.

So by now, you should have figured out why we call our searches “Analytic Search”…

I hope you will test it (we have a free license) and gain value from it.

You are welcome to let us know what you think