Apache Error Log & Apache Access Log:
Analyzing + Troubleshooting Apache
(Updated Guide 2020)
+ Bonus: 20 Apache errors – a free checklist
Apache error logs and Apache access logs contain valuable data.
In this article, we explain how the log files generated by the Apache web server are an important factor in keeping your web sites and apps running 24/7.
We show you how to effectively use Apache logs to monitor and troubleshoot Apache log files, to protect and fix your web server.
Let’s begin
A small anecdote – Did you know the Apache web server was named after a real Native American tribe?
Just like Akecheta in HBO’s Westworld TV show, Apache web server has followed a similar path to the show’s artificial warrior; going from what was jokingly referred to as “a patchy server” to the dominant web server on the internet.
Just like Apache web server, the life of brave was more than just throwing tomahawks, and smoking peace pipes, It also consisted of a myriad of boring day to day tasks.
Bottom line: The job of a web server is more than just delivering web content but also ensuring that your web server stays up and running, requires constant vigilance, monitoring, and maintenance.
What is the Apache web server?
Before we dive into the specifics of Apache, let’s define what we mean by the term Web Server.
At its heart, a web server is basically a piece of software that receives requests and responds by sending back data or files.
These requests are sent by web browsers or other application using the Hypertext Transfer Protocol (HTTP), and the web server responds using the same protocol to return data in a range of format, including text, XML, JSON, JPG, PNG, and various video formats.
Apache is an open source web server that is used to power more than half the sites on the world wide web.
Apache owes its dominance to being one of the first available products in this space, and has become the default platform provided by that other open source giant of the internet, the Linux operating system.
Apart from being the OG Internet Server, Apache has become an essential tool for website builders due to a number of factors.
Under the hood, Apache is based on a flexible and highly modular design.
Note: This lets you extend Apache’s basic functionality to meet your specific needs from a large catalog of existing modules.
In addition, Apache’s thread-based architecture and support for virtual hosts enable it to support multiple websites on a single machine.
Access logs and error logs – logging basics
Let’s look at the logging tools Apache provide out of the box, these are the Access Log, and the Error log.
The type of data Apache collects –
Apache access logs: as its name suggests, the access log collects data relating to external parties, such as web browsers or other types of software trying to access your web server.
Specifically, this log records incoming HTTP requests and the status of the server’s response.
This makes it an important resource for analyzing site traffic, and trying to locate potential attempts to breach your site’s security.
Apache error log: likewise, the Error log also does exactly what it says on the tin, and records a list of errors that occurred as Apache tried to respond to incoming requests.
These can include bugs in web-based software, broken links, or missing files.
We have compiled a free checklist of 20 Apache error messages and their simple explanation, which you can get directly into your mail.
Next step – locate where logged data is stored
This simple task is made harder by the fact that Apache is cross-platform software that runs on nearly every server-based operating system.
Learn how the Apache log analysis app makes it easy for you.
While you might expect that Apache log files will be stored in different file system locations when running on a Mac or Windows PC, you should note that different Linux distros have their own ideas where to store log data.
Apache also stores data in a number of different logging formats.
The format your Apache installation uses is defined in the Apache configuration file.
- By default, Apache logs data uses the Common Log Format – the Common Log Format defines a label/nickname and associates it.
- Another popular format is the Combined Log Format – this format is based on the Common Log Format, and combines it with the header of the HTTP access request.
- Apache also lets you store log data in many common data formats, such as XML and JSON.
Extracting data from Apache error logs and Apache access logs
Before exploring how to use log files, we need to discuss the best way to extract data from them.
Apache access and error logs can give us valuable insights into the usage and functioning of an Apache-based website or application.
However, the sheer amount of data that is collected can be overwhelming, and finding important indicators can be a daunting process.
The obvious way to read log file data is using a basic text editor, but many editors may have a hard time dealing with large log files.
- A common approach to reading log files, especially on Linux/Unix based systems, is to take advantage of the operating system’s command line tools, such as GREP and AWK.
- Another approach is to use scripting languages, such as BASH or Powershell to parse log files.
Alternatively, you can use a log management tool such as XpoLog that automatically collects, manages, and analyzes your Apache data – Download free.
Apache access logs – what to investigate?
Apache access logs give us detailed information about external requests to access our web sites.
Using the log, we can retrieve data related to:
- The remote client (IP, DNS).
- The identity of the user who made the request.
- The HTTP status code of the request.
- The date and time the request was made (timestamp).
In addition to this information, the access log gives us detailed information from the user agent that made the request.
- This tells us about the application, usually browser, that sent the request, as well as information relating to the host device, such as its operating system.
- Access logs indicate the location and source from which server requests originated.
All this information can be aggregated, and reveal much about your site’s user and traffic.
Access logs and suspicious/malicious behavior detection.
You could use the Access logs to indicate the number of requests made to restricted parts of your site.
This is a good indicator of bad actors trying to gain access to confidential information.
An unusually high number of access requests coming from a single source, or multiple random sources could be a sign that your site is being subjected to a distributed denial of service (DoSS) attack.
Access logs and problems diagnosis.
Using access logs, you can detect problems such as problematic API calls, and missing file errors.
Access logs and web server performance monitoring.
By aggregating performance information, such as the time it takes for your web applications to respond and process the requests, this will see poorly performing application code, and scaling issues.
Apache Error Logs – Troubleshooting Common Issues
Apache error logs let you locate and resolve web server related issues.
The error logs reports errors that relate to:
- Configuration.
- Execution.
- Files.
- Permissions.
- Proxies.
- And session caching.
Detect specific low-level errors/diagnose system-wide problems.
- Reported errors can indicate if a client was denied access to a URL or file because it did not conform to configuration file parameters.
- It can also indicate that a proxy agent tried to access the server using an invalid URL.
- They can also show if a file a client tried to access actually exists or was not available.
Find individual problems and configure error logging to provide verbose error logs.
You should only use verbose setting when the total number of reported errors is in a specific area, or relatively low, otherwise the amount of reported data could cause performance issues.
Determine how your web server deals with system-wide problems.
For example, if your web server reports an unexpectedly large number of different errors, this may be an indication that there is a fundamental problem with your system.
Another common problem that can be detected from Apache error logs, relates to the number of times your system starts and stops.
If your system is continually rebooting due to system errors, you will probably need to launch a detailed investigation.
Apache error log & Apache access log automation:
Analyzing and tracking Apache logs in minutes.
Apache’s log files provide a wealth of information, divided between access logs and error logs.
- Access logs collect data relating to external parties, such as web browsers or other types of software trying to access your web server.
- Error logs record the errors that occur as a result of Apache operations, such as bugs in web-based software, broken links, or missing files.
Apache lets you configure these logs to collect as much information, in various levels of detail.
The problem: as useful as this information is, it can be hard to digest on its own, and this may lead you to miss important indicators hidden in these files.
The solution: Use a fully automated tool that collects, parses and normalizes all your log data, suggests a relevant analysis app with ready to use dashboards and reports, such as the Apache app, ads AI problem detection, monitors, search filters and apptags and makes the experience of log analysis and management as simple as it gets!
See How easy it is
Apache Logs – Traffic and Bandwidth Analysis & Monitoring
In order to ensure that your web sites are usable, XpoLog gives you an accurate picture of how many people are trying to access them.
This not only gives you an immediate way to measure the size of your audience, it also shows you how much network resources you need and the bandwidth you need to support current and future needs.
XpoLog’s Bandwidth dashboard shows you where your traffic originated, and how much bandwidth that location required.
In addition, the Apache app presents the total quantity of data consumed by each of your users.
Apache Logs – Geo Data Analysis
XpoLog’s Geo Data dashboard provides a number of ways of looking at your Apache access log data in its geographic context.
The Visitors By Geo Data gadget maps your audience by country.
The Apache app also provides a pie chart that divides your users according to their location, by numbers and percentage.
The dashboard not only shows where your users are located, but also shows you usage patterns over time, and which URLs are being accessed from which places.
Apache Logs – Operating System and Browser Statistics
Another set of important Apache metrics that XpoLog surfaces give you important insights into how your audience access your web site and consumes its information.
Operating systems and devices including: desktop computers, laptops, phones, tablets, game console, and watches.
Anything from a device’s graphics hardware to its browser implementation can cause subtle or dramatic changes to the way your content is presented.
In order to overcome these challenges, XpoLog allows you to understand the myriad of ways and devices used by your audience.
View a complete picture of how your site is browsed – browser and OS statistics.
XpoLog provides detailed browser error and status data.
XpoLog also aggregates access data across each platform.
Apache logs – HTTP Errors
Get a complete picture of what your Apache server is doing.
XpoLog shows you both the number of errors and the URLs that cause them.
XpoLog also shows you on which of your servers a problem occurs, it also maps which locations are experiencing errors.
Apache Log Analysis – Just the Tip of The Iceberg
There is far more Apache data captured and analyzed by XpoLog than we have time or space to present.
XpoLog also shows performance, server, user statistics and much more.
- XpoLog streams live data.
- XpoLog analyzes the data and visualizes it in a manner that gives you actionable insights.
- Add monitors in a click or create your own.
- Use XpoLog’s AI-powered engine to proactively detect errors and problems.
- XpoLog offers the quickest and simplest log search experience.
Make your life super easy 🙂
