wgtool/scripting/README.md

NPM Log Analysis Tools

High-performance security analysis tools for NPM (Nginx Proxy Manager) logs.

Overview

This repository contains two versions of the NPM log analyzer:

1. Bash Version (npm-log-analyzer.sh) - Interactive menu-driven tool
2. Go Version (npm-log-analyzer.go) - High-performance command-line tool

Features

Security Pattern Detection

• Critical Attacks: SQL injection, XSS, shell/RCE, webshell uploads
• High Priority: Path traversal, WordPress hunting, backup harvesting
• Reconnaissance: Robots.txt requests, vulnerability scanners, error spam
• Advanced: SSRF, LFI/RFI, deserialization, template injection

Analysis Capabilities

• Real-time log processing
• IP address analysis and geolocation
• Attack pattern counting and categorization
• Comprehensive reporting
• Performance optimization for large log files

Quick Start

Go Version (Recommended)

# Build and run
make go

# Or manually
go build -o npm-log-analyzer-go npm-log-analyzer.go
./npm-log-analyzer-go

Bash Version

# Make executable and run
chmod +x npm-log-analyzer.sh
./npm-log-analyzer.sh

Performance Comparison

Feature	Go Version	Bash Version
Speed	~85 seconds for 260MB logs	~2-3 minutes
Memory	Efficient streaming	Higher memory usage
Features	Command-line focused	Interactive menu
Dependencies	Single binary	Requires bash, grep, etc.
Gzip Support	✅ Native	❌ Limited

Recent Analysis Results

From the latest Go analysis (260MB of logs):

Critical Findings

• SQL Injection Attempts: 378
• Shell/RCE Attempts: 2,693
• Error Spam (404/403): 4,855

Top Attack Sources

• 169.150.203.13: 45,278 requests (suspicious high volume)
• 135.181.143.221: 9,229 requests
• 97.120.203.58: 3,405 requests

Analysis Statistics

• Processed Files: 98
• Total Lines: 105,568
• Unique IPs: 959
• Duration: ~85 seconds

Configuration

Log Directory

Both tools expect NPM logs at: /opt/stacks/npm/data/logs

Output Directory

Reports are saved to: ./npmlogs

Attack Patterns

Patterns are defined in the source code and can be customized:

// Go version
var AttackPatterns = map[string]string{
    "sql_injection": `union|select|insert|drop|delete`,
    "xss":          `<script|javascript:|onload=|onerror=`,
    // ... more patterns
}

# Bash version
declare -A ATTACK_PATTERNS=(
    ["sql_injection"]="union|select|insert|drop|delete"
    ["xss"]="<script|javascript:|onload=|onerror="
    # ... more patterns
)

Usage Examples

Go Version

# Quick analysis
./npm-log-analyzer-go

# Build for different platforms
GOOS=linux GOARCH=amd64 go build -o npm-analyzer-linux npm-log-analyzer.go

Bash Version

# Interactive menu
./npm-log-analyzer.sh

# Quick analysis only
echo "1" | ./npm-log-analyzer.sh

Makefile Targets

make build-go      # Build Go version
make run-go        # Build and run Go version
make go            # Alias for run-go
make run-bash      # Run Bash version
make bash          # Alias for run-bash
make benchmark     # Compare performance
make clean         # Clean build artifacts
make help          # Show all targets

Requirements

Go Version

• Go 1.16+ (for native gzip support)
• No external dependencies

Bash Version

• Bash 4.0+
• grep, find, awk, curl
• Read access to /opt/stacks/npm/data/logs

Security Considerations

1. Log Access: Both tools require read access to NPM logs
2. Network Access: IP geolocation requires internet access (optional)
3. File Permissions: Output directories need write permissions
4. Large Files: Processing 260MB+ logs requires sufficient memory

Troubleshooting

Common Issues

"Log directory not accessible"

# Check if NPM is running
ls -la /opt/stacks/npm/data/logs/

# Check permissions
sudo ls -la /opt/stacks/npm/data/logs/

"No output generated"

• Large log files take time to process
• Go version is much faster for large files
• Check available memory and disk space

"Permission denied"

# Make scripts executable
chmod +x npm-log-analyzer.sh
chmod +x npm-log-analyzer-go

# Check output directory permissions
mkdir -p ./npmlogs
chmod 755 ./npmlogs

Development

Adding New Patterns

1. Edit the pattern definitions in the source code
2. Test with sample log data
3. Update documentation

Performance Optimization

• Go version uses buffered I/O and streaming
• Bash version uses grep with timeouts
• Both versions filter internal IPs automatically

License

This project is open source. Feel free to modify and distribute.

Contributing

1. Test both versions with your log data
2. Report any issues or performance problems
3. Suggest new attack patterns or features
4. Submit pull requests for improvements