docker-tools/bash/DEBUGGING.md

# Debugging and Optimization Guide

This document describes the debugging and optimization features added to the docker-tools bash scripts.

## Debug Mode

Enable comprehensive debugging with the `--debug` or `-d` flag:

```bash
./docker-backup.sh --debug --all
./docker-migrate.sh --debug --service myapp --dest user@host
```

### What Debug Mode Does

1. **Enables `set -x` (xtrace)**: Shows every command executed with line numbers
2. **Function tracing**: Shows function entry points with file:line information
3. **Stack traces**: On errors, shows call stack
4. **Detailed logging**: All operations log debug information
5. **Verbose output**: Shows internal state and decisions

### Debug Output Format

```
+ [script.sh:123] function_name(): Command being executed
[DEBUG] script.sh:123 function_name()
[DEBUG] Validating stack name: 'myapp'
[DEBUG] Stack name validation passed
```

## Verbose Mode

Enable verbose logging without full xtrace:

```bash
./docker-backup.sh --verbose --all
```

Verbose mode provides:
- Function entry/exit logging
- Operation details
- State information
- Without the command-level tracing of debug mode

## Debug Logging Functions

### `log_debug()`
Logs debug messages (only shown in DEBUG or VERBOSE mode):
```bash
log_debug "Processing stack: $stack"
```

### `log_verbose()`
Logs verbose messages (only shown in VERBOSE mode):
```bash
log_verbose "Found ${#stacks[@]} stacks"
```

### `debug_trace()`
Automatically called in functions to show entry points:
- Shows file, line number, and function name
- Only active when DEBUG or VERBOSE is enabled

## Error Handling Improvements

### Better Error Messages
- More descriptive error messages with context
- Shows what was attempted and why it failed
- Includes relevant variable values

### Exit Codes
- Consistent exit codes across scripts
- `0`: Success
- `1`: General error
- `2`: Invalid arguments
- `3`: Dependency missing

### Error Stack Traces
In DEBUG mode, errors show call stack:
```
[ERROR] Stack directory not found: /opt/stacks/myapp
  -> docker-backup.sh:248 backup_stack()
  -> docker-backup.sh:510 main()
```

## Performance Optimizations

### Reduced Redundant Operations
- Docker commands cached where appropriate
- Stack discovery optimized
- Parallel operations properly throttled

### Improved Logging
- Log file writes are non-blocking
- Fallback to /tmp if primary log location fails
- Automatic log directory creation

### Spinner Optimization
- Spinner disabled in DEBUG mode (would interfere with output)
- Proper cleanup on exit

## Common Debugging Scenarios

### Stack Not Found
```bash
DEBUG=true ./docker-backup.sh --stack myapp
# Shows:
# [DEBUG] Discovering stacks in: /opt/stacks
# [DEBUG] Found directory: myapp
# [DEBUG] Validating stack name: 'myapp'
# [ERROR] Stack directory not found: /opt/stacks/myapp
```

### Docker Connection Issues
```bash
DEBUG=true ./docker-backup.sh --all
# Shows:
# [DEBUG] Checking Docker installation and daemon status
# [DEBUG] Docker command found: /usr/bin/docker
# [DEBUG] Docker version: Docker version 24.0.5
# [ERROR] Docker daemon not running or not accessible.
# [DEBUG] Attempting to get more details...
```

### Transfer Failures
```bash
DEBUG=true ./docker-migrate.sh --service myapp --dest user@host
# Shows:
# [DEBUG] Executing SSH command: host=host, port=22, user=user
# [DEBUG] SSH connection: user@host:22
# [DEBUG] SSH command exit code: 255
# [ERROR] SSH connection failed
```

## Environment Variables

You can also set debug mode via environment variables:

```bash
export DEBUG=true
./docker-backup.sh --all

export VERBOSE=true
./docker-migrate.sh --service myapp --dest user@host
```

## Log Files

Debug information is written to log files:
- Default: `/tmp/docwell/docwell.log`
- Script-specific: `/tmp/docwell/docker-{script}.log`
- Fallback: `/tmp/docwell/fallback.log` (if primary location fails)

Log format:
```
2025-02-18 10:30:45 [DEBUG] Validating stack name: 'myapp'
2025-02-18 10:30:45 [INFO] Starting backup for myapp
2025-02-18 10:30:46 [OK] Backup complete: myapp
```

## Best Practices

1. **Use DEBUG for troubleshooting**: When something fails unexpectedly
2. **Use VERBOSE for monitoring**: When you want to see what's happening without full trace
3. **Check log files**: Even without DEBUG, errors are logged
4. **Combine with dry-run**: `--debug --dry-run` to see what would happen
5. **Redirect output**: `--debug 2>debug.log` to save debug output

## Troubleshooting Tips

### Script Hangs
- Enable DEBUG to see where it's stuck
- Check for SSH timeouts or network issues
- Look for spinner processes that didn't clean up

### Permission Errors
- DEBUG shows exact commands being run
- Check if sudo is needed (shown in debug output)
- Verify file/directory permissions

### Unexpected Behavior
- Compare DEBUG output with expected behavior
- Check environment variables (shown at script start)
- Verify configuration file values

## Performance Monitoring

With VERBOSE mode, you can see:
- How many stacks were discovered
- Time spent on each operation
- Parallel job status
- Resource usage

Example:
```bash
VERBOSE=true ./docker-backup.sh --all
# Shows:
# [VERBOSE] Found 5 directories, 5 valid stacks
# [VERBOSE] Total stacks discovered: 5 (0 from docker ps)
# [VERBOSE] Processing stack 1/5: myapp
```