Log Analysis

When to use this skill

Error debugging

analyze the root cause of application errors

Performance analysis

analyze response times and throughput

Security audit

detect anomalous access patterns
Incident response: investigate the root cause during an outage Instructions Step 1: Locate Log Files

Common log locations

/var/log/

System logs

/var/log/nginx/

Nginx logs

/var/log/apache2/

Apache logs

./logs/

Application logs

Step 2: Search for Error Patterns Common error search :

Search ERROR-level logs

grep -i "error|exception|fail" application.log

Recent errors (last 100 lines)

tail -100 application.log | grep -i error

Errors with timestamps

grep -E "^[.*ERROR" application.log HTTP error codes :

5xx server errors

grep -E "HTTP/[0-9.]+ 5[0-9]{2}" access.log

4xx client errors

grep -E "HTTP/[0-9.]+ 4[0-9]{2}" access.log

Specific error code

grep "HTTP/1.1 \" 500" access.log Step 3: Pattern Analysis Time-based analysis :

Error count by time window

grep -i error application.log | cut -d ' ' -f1,2 | sort | uniq -c | sort -rn

Logs for a specific time window

grep "2025-01-05 14:" application.log IP-based analysis :

Request count by IP

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20

Activity for a specific IP

grep "192.168.1.100" access.log Step 4: Performance Analysis Response time analysis :

Extract response times from Nginx logs

awk '{print $NF}' access.log | sort -n | tail -20

Slow requests (>= 1 second)

awk '$NF > 1.0 {print $0}' access.log Traffic volume analysis :

Requests per minute

awk '{print $4}' access.log | cut -d: -f1,2,3 | uniq -c

Requests per endpoint

awk '{print $7}' access.log | sort | uniq -c | sort -rn | head -20 Step 5: Security Analysis Suspicious patterns :

SQL injection attempts

XSS attempts

grep -iE "<script|javascript:|onerror=" access.log

Directory traversal

grep -E "../" access.log

Brute force attack

grep -E "POST.*/login" access.log | awk '{print $1}' | sort | uniq -c | sort -rn Output format Analysis report structure

Log analysis report

Summary

Analysis window: YYYY-MM-DD HH:MM ~ YYYY-MM-DD HH:MM

Total log lines: X,XXX

Error count: XXX

Warning count: XXX

|

| | Error A | 150 | 2025-01-05 14:30 | | Error B | 45 | 2025-01-05 14:25 |

Recommended actions

1.

[Action 1]

2.

[Action 2]

Best practices

Set time range

clearly define the time window to analyze

Save patterns

script common grep patterns

Check context

review logs around the error too (
-A
,
-B
options)
Log rotation: search compressed logs with zgrep as well Constraints Required Rules (MUST) Perform read-only operations only Mask sensitive information (passwords, tokens) Prohibited (MUST NOT) Do not modify log files Do not expose sensitive information externally References grep manual awk guide Log analysis best practices Examples Example 1: Basic usage Example 2: Advanced usage

log-analysis

安装