Log search – How to Run Simple Log Searches in XpoLog Center 7

CHAPTER 8

Once data is ingested into XpoLog, you can search it for important events, patterns, and anomalies.

XpoLog Center 7 has a powerful search engine that allows users to write and run log search queries from the console using an easy-to-use language.

With XpoLog log search, you can also:

Create monitors that run on the log data at regular intervals and send alerts when a search pattern is found (log monitors and are discussed in the next chapter). Learn more about XpoLogs’ log monitoring tool here.
View the distribution of the occurrences of searched data in a timeline.
Save log searches for later use.
Share log search queries and results through popular channels like Slack or e-mail.
Build “gadgets” or visual representations of the search results which can be used in existing apps or new dashboards.
Export search results to CSV or PDF files.

There are two types of log searches you can build in XpoLog: Simple Search and Complex Log Search.

Read more about XpoLogs’ complex log search queries and advanced data mining.

This chapter will show how to write and run simple log searches.

Accessing XpoLog Log Search

To access the Log Search tool, click on “Search” in the top menu.

This brings up the search tool:

You can write your queries in the search box at the top of this page (it is marked with an orange bar and a search icon).

The links in the left navigation pane shows you:

A history of the searches you have run
Any searches you have saved
Documentation on simple search keywords and syntax
Documentation on complex search keywords and syntax

The time picker allows you to narrow down the scope of the search by choosing a time interval.

The default is the last 24 hours. You can choose any of the preset time windows or specify a custom window by choosing the “Custom” option.
To search into streaming logs in real time, you can choose the “Live” option.

TIP: XpoLog recommends you define your search time window before running any search query. This saves XpoLog from spending extra processing power and time to filter out unnecessary events.

Clicking on the “SEARCH” button will run the search.

Writing Simple Log Searches

To search for specific terms with a simple query, you can follow these rules:

Specify where to find the log data. This can be: the log file name, the folder where the file resides, hostname, AppTag
Use boolean operators (AND, OR, NOT) to specify multiple search conditions
Use parentheses to specify the order of search (conditions within parentheses will be evaluated first)
Use double quotations around the searched term for exact matches. This is useful when the searched term contains reserved symbols or keywords
Use UNIX-like wildcards (“?” and “*”) to specify one or more unknown characters in the search terms
To search within a file, folder, host or AppTag, use the “in” operator.

For example: “my search term” in log.my_log_file_name

To search for a value contained within a specific field in the log, use the “contains” operator.

For example: field_name contains “my search term”

To search for an exact match of the value in a specific field in the log, use the equality operator.

For example: field_name=“my search term”

Running a Simple Log Search

For this demonstration, two Apache HTTP server access log files are used.

These files have been added to XpoLog under a folder called “Apache HTTP Logs”.

The files are static, which means they are not being updated.

To search for error 404 in any of the log files, you can run a search like this:

404 in folder.Apache HTTP Logs

The image below shows there are more than 4,600 events in the logs that have the string “404” in it.

Note that the timeframe has been chosen as “All Time” to search through all items in the files.

The graph shows the distribution of the events with the string “404” in them.

Below that, XpoLog shows the matching log records.

To narrow down the search, you can modify the query to something like this:

ResponseStatus=404 in folder.Apache HTTP Logs

This time, the query returns 36 records.

In the left side of the screen, the “Isolate by Sources” section shows only one log file is used. Clicking on the “Logs” link will show a pop-up window listing the “Apache Access Log 1” file only.

The graph shows the distribution of events over the time period chosen (in this case all time):

Holding the mouse button and dragging over this graph shows a magnified view of the event distribution:

As the bell curve shows here, the 404 events occurred between March 9 and March 11.

Under the event distribution graph, the actual log entries will be displayed in the bottom half of the screen (not shown here).

Clicking any field value in an event entry allows the user to add or exclude that value as a search condition or use that as a new search condition as shown below:

In the upper right corner of the screen, there are few options as well:

The distribution graph can be shown as a pie chart, bar chart or line chart from here. By default, the line chart is selected.
The search results can be exported as a CSV or PDF file. The output file will contain matching records from the Apache access logs
The search can be shared with other users in the enterprise. The “Share Search” drop-down menu shows the channels that can be used for sharing (for example clipboard, Slack or e-mail)
The search can be saved for later use. Saving a search involves giving it a name, a description, the default time range, and a severity level (The severity level ensures the saved search is used as a predefined rule by the Analytics engine). The search query is automatically selected:

The search can be used to create a monitor. The image below shows the fields needed to create the monitor:

Finally, the visual representation of the search result can be used to create a gadget. In the image below, a chart type gadget is being created. The dialog box also allows creating a new app from the gadget and placing the gadget in a dashboard of that app:

Clicking on “SAVE AND VIEW” will show the app opened in a new browser tab. Clicking on the “Apache HTTP Responses” dashboard will show the gadget: