Log search – How to Run Simple Log Searches in XpoLog Center 7

CHAPTER 8

log analyzer that adds machine intelligence automated problem discovery

Once data is ingested into XpoLog, you can search it for important events, patterns, and anomalies.

XpoLog Center 7 has a powerful search engine that allows users to write and run log search queries from the console using an easy-to-use language.

With XpoLog log search, you can also:

  • Create monitors that run on the log data at regular intervals and send alerts when a search pattern is found (log monitors and are discussed in the next chapter). Learn more about XpoLogs’ log monitoring tool here.
  • View the distribution of the occurrences of searched data in a timeline.
  • Save log searches for later use.
  • Share log search queries and results through popular channels like Slack or e-mail.
  • Build “gadgets” or visual representations of the search results which can be used in existing apps or new dashboards.
  • Export search results to CSV or PDF files.

There are two types of log searches you can build in XpoLog: Simple Search and Complex Log Search.

This chapter will show how to write and run simple log searches.

Accessing XpoLog Log Search

To access the Log Search tool, click on “Search” in the top menu.

how to perform a simple log search in XpoLog

This brings up the search tool:

write and run search queries from the console using an easy-to-use language

You can write your queries in the search box at the top of this page (it is marked with an orange bar and a search icon).

The links in the left navigation pane shows you:

  • A history of the searches you have run
  • Any searches you have saved
  • Documentation on simple search keywords and syntax
  • Documentation on complex search keywords and syntax

The time picker allows you to narrow down the scope of the search by choosing a time interval.

  • The default is the last 24 hours. You can choose any of the preset time windows or specify a custom window by choosing the “Custom” option.
  • To search into streaming logs in real time, you can choose the “Live” option.

narrow down the scope of the search by choosing a time interval

TIP: XpoLog recommends you define your search time window before running any search query. This saves XpoLog from spending extra processing power and time to filter out unnecessary events.

  • Clicking on the “SEARCH” button will run the search.

Writing Simple Log Searches

To search for specific terms with a simple query, you can follow these rules:

  • Specify where to find the log data. This can be: the log file name, the folder where the file resides, hostname, AppTag
  • Use boolean operators (AND, OR, NOT) to specify multiple search conditions
  • Use parentheses to specify the order of search (conditions within parentheses will be evaluated first)
  • Use double quotations around the searched term for exact matches. This is useful when the searched term contains reserved symbols or keywords
  • Use UNIX-like wildcards (“?” and “*”) to specify one or more unknown characters in the search terms
  • To search within a file, folder, host or AppTag, use the “in” operator.

For example: “my search term” in log.my_log_file_name

  • To search for a value contained within a specific field in the log, use the “contains” operator.

For example: field_name contains “my search term”

  • To search for an exact match of the value in a specific field in the log, use the equality operator.

For example: field_name=“my search term”

Running a Simple Log Search

For this demonstration, two Apache HTTP server access log files are used.

These files have been added to XpoLog under a folder called “Apache HTTP Logs”.

The files are static, which means they are not being updated.

Apache log search

To search for error 404 in any of the log files, you can run a search like this:

404 in folder.Apache HTTP Logs

The image below shows there are more than 4,600 events in the logs that have the string “404” in it.

Note that the timeframe has been chosen as “All Time” to search through all items in the files.

search for error 404 in any of the log files

The graph shows the distribution of the events with the string “404” in them.

Below that, XpoLog shows the matching log records.

To narrow down the search, you can modify the query to something like this:

ResponseStatus=404 in folder.Apache HTTP Logs

This time, the query returns 36 records.

In the left side of the screen, the “Isolate by Sources” section shows only one log file is used. Clicking on the “Logs” link will show a pop-up window listing the “Apache Access Log 1” file only.

Isolate by Sources

The graph shows the distribution of events over the time period chosen (in this case all time):

The graph shows the distribution of events over the time period chosen

Holding the mouse button and dragging over this graph shows a magnified view of the event distribution:

Holding the mouse button and dragging over this graph shows a magnified view of the event distribution

As the bell curve shows here, the 404 events occurred between March 9 and March 11.

Under the event distribution graph, the actual log entries will be displayed in the bottom half of the screen (not shown here).

Clicking any field value in an event entry allows the user to add or exclude that value as a search condition or use that as a new search condition as shown below:

add or exclude that value as a search condition or use that as a new search condition

In the upper right corner of the screen, there are few options as well:

more options in the log search tool

  • The distribution graph can be shown as a pie chart, bar chart or line chart from here. By default, the line chart is selected.
  • The search results can be exported as a CSV or PDF file. The output file will contain matching records from the Apache access logs
  • The search can be shared with other users in the enterprise. The “Share Search” drop-down menu shows the channels that can be used for sharing (for example clipboard, Slack or e-mail)
  • The search can be saved for later use. Saving a search involves giving it a name, a description, the default time range, and a severity level (The severity level ensures the saved search is used as a predefined rule by the Analytics engine). The search query is automatically selected:

Saving a search involves giving it a name, a description, the default time range, and a severity level (The severity level ensures the saved search is used as a predefined rule by the Analytics engine)

  • The search can be used to create a monitor. The image below shows the fields needed to create the monitor:

The log search can be used to create a monitor

  • Finally, the visual representation of the search result can be used to create a gadget. In the image below, a chart type gadget is being created. The dialog box also allows creating a new app from the gadget and placing the gadget in a dashboard of that app:

the visual representation of the search result can be used to create a gadget

  • Clicking on “SAVE AND VIEW” will show the app opened in a new browser tab. Clicking on the “Apache HTTP Responses” dashboard will show the gadget:

Save log search visualization as a gadget and turn it into an app

Conclusion

This has been a high-level walkthrough of using XpoLog Log Search tool.

Once familiar with basic queries, users can start writing more advanced queries to fine-tune their analysis and create more meaningful insights.

Need a demo?

All chapters: 

Read more about:

How to secure Amazon S3 buckets
View how raw data from the Champions League easily turn into insights
Troubleshoot your Windows task scheduler in 5 minutes! XPLG automated log analyzer and problem detection