Read Apache and Nginx Logs to Diagnose 4xx–5xx Errors

If you run a website on Apache or Nginx, access and error logs are a useful starting point for investigating 404, 403, 500 and 502 responses. What they capture depends on logging configuration and which layer handled the request; an error returned by a CDN or proxy may not appear in the origin server’s logs. Learning to read the available request paths, addresses, status codes and timestamps gives you concrete clues for investigating those failures.

In this guide we’ll walk through how to read Apache and Nginx access and error logs with a focus on diagnosing 4xx–5xx HTTP errors. We’ll look at illustrative log examples, explain what each field means in plain language, and show practical command‑line queries you can copy‑paste. Whether you are on shared hosting, a VPS, or a dedicated server at dchost.com, the general workflow is the same, subject to the log access your plan provides: find the right log file, filter for problematic status codes, and correlate entries with other logs to investigate the cause.

Why 4xx–5xx Diagnosis Starts with Your Web Server Logs

HTTP status codes describe the outcome reported by the layer responding to a request. 2xx means success, 3xx covers redirection and cache validation, 4xx indicates a perceived client error, and 5xx indicates a server error. Browser error pages are only a user-facing summary; web server, proxy and application logs may provide the details needed to investigate.

When the affected requests come from Googlebot, “Googlebot 5xx Errors: Diagnosing Hosting-Related Crawl Failures” explains how to match Search Console findings with server logs and isolate the failing layer.

Logs can provide clues to questions such as:

• “Are these 404s caused by bots, broken internal links, or old external links?” → Check reported referrer and user agent in access logs, treating them as clues rather than proof.
• “Why are we getting random 500 errors on checkout?” → Correlate 500s in access logs with PHP or application errors in error logs.
• “Is that 503 from the app, web server, or proxy?” → Compare logs from those layers with maintenance settings, upstream health and resource metrics; the status alone does not establish throttling.

We already covered what the main HTTP codes mean for SEO and redirects in our article What HTTP Status Codes Mean for SEO and Hosting. Here we’ll focus on reading raw web-server log lines for 4xx and 5xx responses, then correlating them with application logs and system metrics to narrow down the cause.

Quick Refresher: HTTP 4xx and 5xx Status Codes

Before diving into logs, it helps to have a mental map of the most common 4xx–5xx codes you’ll see.

Common 4xx (Client Error) Codes

• 400 Bad Request – The server rejects the request as invalid (for example, malformed headers or broken syntax). Additional details may appear in error logs.
• 401 Unauthorized – Authentication required or failed. A conforming response must include a WWW-Authenticate challenge.
• 403 Forbidden – The server understood the request but refuses to authorize it. Common with directory restrictions or security rules.
• 404 Not Found – The server cannot find a current representation of the requested resource, or is unwilling to disclose that it exists. Missing URLs and incorrect rewrite rules are common causes.
• 405 Method Not Allowed – Method (POST, PUT, DELETE) is not allowed for this endpoint.
• 408 Request Timeout – Client took too long to send the full request.
• 429 Too Many Requests – Rate limiting by web server, WAF or application.

Common 5xx (Server Error) Codes

• 500 Internal Server Error – Generic error; often PHP, application or configuration failure.
• 502 Bad Gateway – Web server (reverse proxy) couldn’t get a valid response from upstream (PHP‑FPM, Node.js, etc.).
• 503 Service Unavailable – Server is overloaded, under maintenance, or backend pool is down.
• 504 Gateway Timeout – Upstream took too long to respond; often a slow database or external API.

Your Apache or Nginx access log can record these responses when access logging is enabled for the requests concerned. The error log may explain why the server returned a code, but not every error response has a corresponding error-log entry. Reading both together is the key skill you want to build.

Where to Find Apache and Nginx Logs on Common Systems

Exact paths depend on your OS, control panel and web server configuration. The following are common Linux locations.

Typical Apache Log Locations

• Debian/Ubuntu
  /var/log/apache2/access.log
/var/log/apache2/error.log
• CentOS / AlmaLinux / Rocky Linux
  /var/log/httpd/access_log
/var/log/httpd/error_log
• Per‑vHost logs (common on panels)
  Something like /home/USER/logs/domain.com-access_log and domain.com-error_log

Typical Nginx Log Locations

• Common Linux package locations
  /var/log/nginx/access.log
/var/log/nginx/error.log
• Per‑site logs (based on your vhost config)
  Example snippet (requires a log format named main, as shown below):
  access_log /var/log/nginx/example.com.access.log main;
error_log /var/log/nginx/example.com.error.log warn;

On shared hosting, logs may be available through a control panel or as per‑domain files inside your home directory. On a VPS or dedicated server, log access depends on the permissions and management arrangement; confirm whether your chosen plan provides root or delegated log access. If you’re planning a move from shared hosting to a VPS primarily for better observability and control, our checklist for moving from shared hosting to a VPS with zero downtime is a good companion read.

Understanding Access Log Formats (With Illustrative Examples)

Common access-log formats include Common Log Format (CLF) and Combined Log Format, though deployments may use custom formats. Combined adds referrer and user agent, which can provide useful context for 4xx–5xx diagnosis.

Apache Combined Log Format

Definition (from LogFormat):

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"

Example line:

203.0.113.45 - - [05/Dec/2025:10:23:12 +0000] "GET /checkout HTTP/1.1" 500 512 "https://example.com/cart" "Mozilla/5.0 ..."

Reading it from left to right:

• 203.0.113.45 – Recorded remote address in this example; hostname lookup and proxy settings can affect this field, so it need not identify the original visitor.
• – – – Unavailable remote identity and authenticated username
• [05/Dec/2025:10:23:12 +0000] – Timestamp, including timezone offset
• “GET /checkout HTTP/1.1” – Request line: method, request target, protocol
• 500 – Status code (here, a server error)
• 512 – Response body size in bytes, excluding headers
• “https://example.com/cart” – Referrer reported by the client
• “Mozilla/5.0 …” – Reported user agent, which may claim to be a browser, bot or app

This illustrative line records a GET request to /checkout that returned 500, with the cart URL reported as the referrer. It does not establish that a person clicked a particular button: referrer and user-agent headers can be omitted or forged. Use the path and timestamp to investigate the request and correlate it with error or application logs.

Nginx Log Format

Nginx lets you define log_format. A common “combined” style format looks like:

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent" "$request_time"';

Example line:

203.0.113.45 - - [05/Dec/2025:10:23:12 +0000] "POST /api/login HTTP/2.0" 401 178 "https://app.example.com/" "Mozilla/5.0 ..." "0.123"

• 401 – Authentication required or failed
• “0.123” – Request processing time in seconds (very handy when debugging slow 5xx responses)

The key for 4xx–5xx diagnosis is simple: spot the 4xx–5xx codes, then inspect the surrounding fields. Check the recorded client address, request, reported referrer and user agent, and request duration where available.

Understanding Error Logs: Diagnostic Messages and Their Limits

Access logs tell you that a request returned 500. Error logs may help explain why, though application logs may be needed when the web server has no matching diagnostic entry. Let’s look at a few common patterns.

Apache Error Log Examples

Illustrative Apache/PHP-FPM error line with a synthetic application fatal-error message; this is not a captured incident. Actual prefixes and formatting vary by runtime and logging configuration. Correlate it with the access log to establish the HTTP status:

[Fri Dec 05 10:23:12.345678 2025] [proxy_fcgi:error] [pid 12345:tid 140123456789] [client 203.0.113.45:51724] AH01071: Got error 'PHP message: PHP Fatal error:  Checkout configuration missing in /home/user/public_html/wp-content/plugins/custom-checkout.php on line 42'

Pieces to notice:

• [proxy_fcgi:error] – The module and severity that logged the error; here Apache’s FastCGI proxy is relaying a PHP-FPM message.
• [client 203.0.113.45:51724] – Client address to compare with the access log; proxy and logging settings can affect which address appears.
• PHP Fatal error… – The reported application failure to investigate; the message may not establish its underlying cause.

Use the timestamp and client address to find candidate requests in access.log, then compare the URL and referrer. Allow for request duration and timestamp precision; use a request ID if both logs record one.

Illustrative Apache authorization denial, a pattern to investigate alongside a logged 403:

[Fri Dec 05 10:31:00.123456 2025] [authz_core:error] [pid 22345:tid 140223456789] [client 198.51.100.77:38210] AH01630: client denied by server configuration: /home/user/public_html/wp-admin/admin-ajax.php

This message records a denial by Apache’s authorization layer, rather than a WordPress authorization decision. If it matches a 403 in the access log, start with Apache authorization directives, including those in .htaccess; this message alone does not identify a WAF rule.

Nginx Error Log Examples

Illustrative timeout while Nginx waits for PHP-FPM response headers. Correlate it with the access log to confirm whether the request ended with a 504:

2025/12/05 10:45:22 [error] 1234#1234: *56789 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 203.0.113.45, server: example.com, request: "GET /report?month=12 HTTP/2.0", upstream: "fastcgi://unix:/var/run/php-fpm.sock", host: "example.com"

Important bits:

• upstream timed out – Nginx didn’t get a response in time.
• request: “GET /report?month=12” – The requested URL for this timeout.
• upstream: “fastcgi://unix:/var/run/php-fpm.sock” – The backend whose response header Nginx was waiting for; this entry alone does not establish the underlying cause.

Illustrative file lookup failure to investigate alongside a 404; check whether the file is missing or the request maps to the wrong path:

2025/12/05 11:10:00 [error] 1234#1234: *57000 open() "/home/user/public_html/wp-content/themes/theme/assets/js/app.js" failed (2: No such file or directory), client: 198.51.100.10, server: example.com, request: "GET /wp-content/themes/theme/assets/js/app.js HTTP/2.0", host: "example.com"

Here Nginx could not find the file at the logged path when the request ran. Check the deployment and path mapping. This differs from a 404 caused by application routing, which might not generate a web-server error log entry at all.

Step-by-Step: Diagnosing Common 4xx Errors from Logs

Let’s walk through concrete workflows for the 4xx errors you’ll see most often.

404 Not Found: Broken Links vs. Bot Noise

First, sample the 404 entries to see which URLs are affected. The following field-based commands assume a combined-style access log with a three-part request line (method, target, protocol), the request target in whitespace-separated field 7 and status in field 9. The referrer query below also assumes the reported referrer occupies field 11. Verify your configured format first; custom formats, incomplete request lines or whitespace within fields need a matching parser.

# Apache
awk '$9 ~ /^404$/' /var/log/apache2/access.log | head

# Nginx (status is field 9 in typical formats)
awk '{ if ($9 == 404) print }' /var/log/nginx/access.log | head

Now, group by URL to find the worst offenders:

awk '$9 == 404 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head

This shows counts and paths: maybe /old-blog-post, /wp-login.php, or random bot paths like /.env.

• If you see many 404s on real pages that used to exist, use a 301 redirect when there is a relevant replacement, or return 410 Gone when removal is permanent and there is no replacement. A 410 is not a redirect. Our article on HTTP status codes, SEO and hosting explains when 301 vs 410 is the right choice.
• If 404s are bot scans for /wp-admin, /.git, /phpmyadmin etc., that’s normal background noise, but you may want to harden security or add WAF rules.

You can also break down 404s by referrer:

awk '$9 == 404 {print $11}' /var/log/apache2/access.log | sed 's/"//g' | sort | uniq -c | sort -nr | head

The logged referrer can suggest internal links to fix or external sources to investigate. It may be absent, reduced by referrer policy, or spoofed, so it is a clue rather than proof of origin.

403 Forbidden: Security Rules or Permissions

For 403s you always want to check both access and error logs.

awk '$9 == 403 {print $1, $4, $5, $7}' /var/log/nginx/access.log | head

Then inspect the corresponding server’s error log around the same timestamp. For the earlier Apache example, search its client address:

grep -F "[client 198.51.100.77" /var/log/apache2/error.log | tail

Common 403 causes:

• Directory or file permissions (e.g. chmod 600 on static assets when the serving process is not the owner and cannot read them).
• .htaccess or Nginx rules that deny certain IPs, paths or user agents.
• ModSecurity / WAF rules blocking suspicious patterns in query strings.

If your 403s are mostly WAF‑related, our deep dive on tuning ModSecurity and OWASP CRS to cut false positives discusses targeted adjustments for false positives while retaining relevant protections.

401 Unauthorized: Login Troubles

For 401 errors, look at:

• Path – Is it an API endpoint, admin panel, or HTTP Basic Auth area?
• User agent – Does it claim to be a browser, script, or integration? This value can be spoofed.
• Rate – Is someone brute‑forcing passwords?

awk '$9 == 401 {print $1, $7, $11}' /var/log/apache2/access.log | sort | head

The command above samples client IP, path, and referrer—not user agent or request rate. If you see many 401s from the same IP, correlate them with authentication logs to distinguish normal challenges or broken integrations from repeated failed logins. Consider temporary blocking or rate limiting only after confirming abuse and accounting for users sharing an IP.

429 Too Many Requests: Rate Limiting in Action

429s indicate that requests are being rejected as too frequent. Check which layer issued them and whether legitimate users are being limited before concluding that the thresholds are appropriate.

awk '$9 == 429 {print $1, $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head

Ask yourself:

• Are 429s mostly from one or a handful of IPs? → Investigate bots, integrations, and shared proxy or NAT addresses; concentration alone does not prove scraping.
• Are real users hitting 429 on checkout or cart APIs? → Limits may be too strict.

Our article on monitoring cart and checkout steps with server logs and alerts discusses using 4xx/5xx patterns to investigate potential customer friction in e‑commerce flows.

Step-by-Step: Diagnosing Common 5xx Errors from Logs

5xx responses report server-side failures and deserve prompt investigation when they affect users or critical workflows. Prioritize by impact, frequency and duration rather than status class alone. Here’s how to narrow down possible causes.

500 Internal Server Error: Application or Config Bugs

Start by listing recent 500s:

awk '$9 == 500 {print $4, $5, $7}' /var/log/apache2/access.log | tail -n 50

Inspect the error log around those times. The next command only shows its last 200 lines; if those do not cover the incident, search the relevant time window in the current or rotated log instead:

tail -n 200 /var/log/apache2/error.log

Typical patterns:

• PHP Fatal error / Uncaught exception – Bug or missing dependency.
• Rewrite rules loop – Misconfigured .htaccess or try_files.
• Permissions problems – App can’t write to cache or storage directories.

If 500s cluster around heavy operations (checkout, reporting, imports), combine this with application and database logs. For PHP apps like WordPress or WooCommerce, our guides on fixing WordPress white screen of death from the hosting side and MySQL/InnoDB tuning for WooCommerce cover troubleshooting and tuning options to investigate alongside the logged cause.

502 Bad Gateway and 504 Gateway Timeout: Upstream Problems

502 and 504 describe gateway failures: an invalid upstream response or an upstream response timeout, respectively. First identify the layer that issued the status—Apache mod_proxy, Nginx, a CDN or another proxy—then investigate its connection to the next upstream service.

Access logs record the status where the request is logged. Error logs at the issuing gateway may add connection or timeout details; correlate those with upstream application logs and health metrics.

• Apache + PHP‑FPM
  Look for socket connection errors. If you also see AH01071: Got error 'Primary script unknown', investigate script-path resolution, but correlate it with the access-log status rather than treating that message alone as evidence of a 502/504.
• Nginx + PHP‑FPM
  Look for upstream timed out or connect() to unix:/var/run/php-fpm.sock failed.

Checklist when you see many 502/504:

• Is the upstream process running? (php-fpm, node, gunicorn, etc.)
• Is the socket/port correct between web server and app?
• Are slow operations keeping HTTP requests open until they time out? Could that work run asynchronously instead?
• Is CPU/IO saturated? (Check top, htop, iostat.)

If resource exhaustion is frequent, it may be time to scale resources or separate app and database servers. Our article on 9 server‑side signals it’s time to upgrade your hosting plan explains symptoms to investigate when deciding whether tuning alone can meet demand.

503 Service Unavailable: Overload or Maintenance

503 can come from either the web server or the application. Look for hints:

• Nginx – Correlate no live upstreams while connecting to upstream with the access-log status and backend health; do not treat this message alone as proof of a 503 or that every backend process has stopped.
• Application – Custom HTML saying “We’re under maintenance” while access logs show 503 but error logs are quiet.

Key questions:

• Do 503s correlate with traffic peaks? → Check for CPU/RAM pressure or database contention.
• Do 503s appear after deploys? → Check whether maintenance mode was left enabled and whether the application started successfully.

Reading logs around traffic spikes alongside system metrics is where proper monitoring pays off. We have a dedicated guide on VPS monitoring and alerts with Prometheus, Grafana and Uptime Kuma that fits neatly on top of the log techniques in this article.

Practical Log Analysis Commands You’ll Actually Use

You don’t need a full SIEM stack to start getting value from logs. A handful of shell commands can help on a Linux server with shell access and permission to read the logs. Use your configured log paths in the examples below. These commands inspect the named files, not all rotated or compressed archives; locate the files covering your incident first. When correlating servers, account for timezone offsets, clock differences and request duration. Restrict log access and redact sensitive addresses, query parameters and other personal or secret data before sharing excerpts.

Follow Logs in Real Time

# Access logs
sudo tail -f /var/log/nginx/access.log

# Error logs
sudo tail -f /var/log/nginx/error.log

Reproduce the failing request in your browser and look for its access-log entry and any corresponding error-log messages. A failed request need not produce an error-log entry.

Filter by Status Code Range (4xx/5xx)

With awk you can quickly filter 4xx and 5xx. The field-based commands below assume a combined-style access log with the status in whitespace-separated field 9 and a three-part request line (method, target, protocol). Custom formats or incomplete request lines need a matching parser.

# All 4xx
awk '$9 ~ /^4[0-9][0-9]$/' /var/log/apache2/access.log | head

# All 5xx
awk '$9 ~ /^5[0-9][0-9]$/' /var/log/nginx/access.log | head

To see the most frequent logged client IPs associated with 5xx responses:

awk '$9 ~ /^5[0-9][0-9]$/ {print $1}' /var/log/nginx/access.log | 
  sort | uniq -c | sort -nr | head

Find Slow Requests that Also Error

The Nginx format illustrated earlier quotes its final $request_time value. Do not run the following numeric comparison directly against logs in that format. It requires status in field 9 and a final, unquoted numeric duration in seconds, without a prefix such as rt=. To use it with the earlier example, first prepare a separate analysis copy that removes only the enclosing double quotes from each final duration field, leaving the other fields unchanged, and point the command at that copy. Verify the resulting fields and keep the original logs intact. Alternatively, use a parser that explicitly decodes the quoted duration; that parser is not supplied here.

# Slow 5xx (request_time > 2 seconds)
awk '$9 ~ /^5/ && $NF > 2 {print $4, $5, $7, $9, $NF}' /var/log/nginx/access.log | head

With those input prerequisites met, this samples 5xx requests lasting more than two seconds. Investigate their application errors and timings before deciding whether optimization, configuration changes or another fix is needed.

Find Error Spikes by Minute

To see if errors come in bursts:

awk '$9 ~ /^5/ {gsub("[:[]"," ",$4); print $4" "$5}' /var/log/apache2/access.log | 
  awk '{print $1" "$2":"$3}' | sort | uniq -c | sort -nr | head

This lists the highest 5xx counts per logged minute in the selected file, which can help you compare incidents with deploys, cron jobs or traffic surges. It counts errors rather than calculating an error rate. Use logs with a consistent timezone for this query, since it does not retain the timezone offset in its grouping.

Turn Logs into Monitoring & Alerts (and When to Upgrade Hosting)

Manual log reading is useful during debugging. For ongoing detection and investigation, consider centralizing logs, choosing retention and creating alerts based on 4xx–5xx patterns. Set retention according to diagnostic needs, storage, privacy and applicable legal requirements; automation supports reliability but does not guarantee it.

Centralised Logging on a VPS

On multi-server architectures, such as separate web, app and database servers, central logging can make correlation easier. Instead of connecting to each machine and searching separately, collect the relevant logs in one place for queries and dashboards. Choose what to collect deliberately, restrict access and avoid shipping unnecessary sensitive data.

Our VPS log management without the drama: centralised logging with Grafana Loki + Promtail guide is a historical reference for centralization, retention and alert design, not a current Promtail deployment recommendation. Promtail reached end of life on March 2, 2026. For new deployments, use Grafana Alloy or another supported client; if you already use Promtail, plan a migration rather than assuming configuration compatibility. The collection and correlation principles apply to a single VPS or multiple dedicated and colocated servers, subject to permissions, resource capacity and storage requirements.

Meaningful Alert Rules Based on 4xx–5xx

Rather than alerting on every isolated 500 across the site, choose rules for your traffic baseline and critical workflows. These thresholds are examples to adapt, not universal defaults:

• Error rate thresholds – e.g. “5xx > 1% of requests for 5 minutes”.
• Path‑specific alerts – e.g. “Any 500/502 on /checkout or /payment callbacks”.
• Spike detection – “404s on a single URL > 100/minute” (often broken campaign links or bot waves).

Combine these with business‑level events (orders per minute, signup success rate) to prioritize user impact and reduce unnecessary alerts.

When Logs Tell You It’s Time to Scale

Logs can suggest a capacity bottleneck, but correlate them with resource metrics and investigate application or configuration faults before deciding to scale. Signs to investigate include:

• Regular 502 errors or 504 timeouts at predictable traffic levels.
• 503s coinciding with CPU at 90–100% and DB load spikes.
• 429 or rate‑limit patterns where you are forced to be overly strict to keep servers alive.

In those cases, optimizing queries and caching is still important, but it’s also time to review your hosting resources (CPU, RAM, NVMe, bandwidth). Our article on capacity planning for WooCommerce in terms of vCPU, RAM and IOPS gives a concrete way to link what you see in logs with what you should provision on a VPS or dedicated server.

Hardening, Performance, and Next Steps

Web server logs are not just for “something is broken” moments. Once you’re comfortable reading 4xx–5xx patterns, you can also use logs to harden security, tune caching, and plan upgrades calmly.

• Security hardening – 403/401/404 patterns can suggest brute-force attempts or path discovery scans, including probes for vulnerable plugins; they do not establish that a targeted plugin is installed or vulnerable. Combine this with a solid baseline from our security checklist for new websites to close obvious doors.
• Performance tuning – High 5xx rates on specific endpoints identify places to investigate. Correlate application errors and timings before attributing failures to slow database queries, missing indexes or lack of caching. Use logs to prioritize which pages to optimize first.
• SEO & UX – Unexpected 404s on pages that should remain available can interrupt user journeys and prevent crawlers from retrieving their content. Use logs alongside Search Console to find broken links and missing pages; a 404 for deliberately removed content is not itself a fault to fix.

A useful starting point is logs first. Before adding hardware or rewriting the application, correlate Apache or Nginx logs with application logs and system metrics. The combination of IP, URL, status, response time and backend error can narrow the investigation.

If you lack the logs needed to investigate recurring 500/502/503 errors, first determine which logs your current setup makes available and which layer generates the failures. If a move is warranted, evaluate log access, retention, monitoring and administrative control alongside CPU and RAM. Plan how web server logs, application logs and metrics will be correlated during an incident.