diff --git a/README.md b/README.md index 2f594543..48bf31d0 100644 --- a/README.md +++ b/README.md @@ -124,6 +124,8 @@ sudo awf --help - [Usage guide](docs/usage.md) — CLI flags, domain allowlists, examples - [Chroot mode](docs/chroot-mode.md) — use host binaries with network isolation - [SSL Bump](docs/ssl-bump.md) — HTTPS content inspection for URL path filtering +- [GitHub Actions](docs/github_actions.md) — CI/CD integration and MCP server setup +- [Environment variables](docs/environment.md) — passing environment variables to containers - [Logging quick reference](docs/logging_quickref.md) and [Squid log filtering](docs/squid_log_filtering.md) — view and filter traffic - [Security model](docs/security.md) — what the firewall protects and how - [Architecture](docs/architecture.md) — how Squid, Docker, and iptables fit together diff --git a/docs/architecture.md b/docs/architecture.md index 95fa95c3..f04a869e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -119,8 +119,8 @@ The wrapper generates: ### 2. Container Startup 1. **Squid proxy starts first** with healthcheck -2. **Copilot container waits** for Squid to be healthy -3. **iptables rules applied** in copilot container to redirect all HTTP/HTTPS traffic +2. **Agent container waits** for Squid to be healthy +3. **iptables rules applied** in agent container to redirect all HTTP/HTTPS traffic ### 3. Traffic Routing - All HTTP (port 80) and HTTPS (port 443) traffic → Squid proxy @@ -136,7 +136,7 @@ The wrapper generates: ### 5. Log Streaming - Container logs streamed in real-time using `docker logs -f` - TTY disabled to prevent ANSI escape sequences -- Copilot and Squid logs preserved to `/tmp/*-logs-/` (if created) +- Agent and Squid logs preserved to `/tmp/*-logs-/` (if created) ### 6. Cleanup - Containers stopped and removed @@ -150,27 +150,27 @@ The wrapper generates: The system uses a defense-in-depth cleanup strategy across four stages to prevent Docker resource leaks: -### 1. Pre-Test Cleanup (CI/CD Scripts) -**Location:** `scripts/ci/test-copilot-*.sh` (start of each script) +### 1. Pre-Test Cleanup (CI/CD) +**Location:** CI/CD workflow scripts **What:** Runs `cleanup.sh` to remove orphaned resources from previous failed runs **Why:** Prevents Docker network subnet pool exhaustion and container name conflicts **Critical:** Without this, `timeout` commands that kill the wrapper mid-cleanup leave networks/containers behind ### 2. Normal Exit Cleanup (Built-in) -**Location:** `src/cli.ts:117-118` (`performCleanup()`) +**Location:** `src/cli.ts` (`performCleanup()`) **What:** - `stopContainers()` → `docker compose down -v` (stops containers, removes volumes) - `cleanup()` → Deletes workDir (`/tmp/awf-`) **Trigger:** Successful command completion ### 3. Signal/Error Cleanup (Built-in) -**Location:** `src/cli.ts:95-103, 122-126` (SIGINT/SIGTERM handlers, catch blocks) +**Location:** `src/cli.ts` (SIGINT/SIGTERM handlers, catch blocks) **What:** Same as normal exit cleanup **Trigger:** User interruption (Ctrl+C), timeout signals, or errors **Limitation:** Cannot catch SIGKILL (9) from `timeout` after grace period ### 4. CI/CD Always Cleanup -**Location:** `.github/workflows/test-copilot-*.yml` (`if: always()`) +**Location:** `.github/workflows/test-*.yml` (`if: always()`) **What:** Runs `cleanup.sh` regardless of job status **Why:** Safety net for SIGKILL, job cancellation, and unexpected failures @@ -194,8 +194,8 @@ Removes all awf resources: ## Exit Code Handling -The wrapper propagates the exit code from the copilot container: -1. Command runs in copilot container +The wrapper propagates the exit code from the agent container: +1. Command runs in agent container 2. Container exits with command's exit code 3. Wrapper inspects container: `docker inspect --format={{.State.ExitCode}}` 4. Wrapper exits with same code diff --git a/docs/logging_quickref.md b/docs/logging_quickref.md index 52004af3..24654163 100644 --- a/docs/logging_quickref.md +++ b/docs/logging_quickref.md @@ -22,7 +22,7 @@ docker exec awf-squid grep "TCP_TUNNEL\|TCP_MISS" /var/log/squid/access.log # From host (requires sudo) sudo dmesg | grep FW_BLOCKED -# From copilot container +# From agent container docker exec awf-agent dmesg | grep FW_BLOCKED # Using journalctl (systemd) @@ -335,5 +335,6 @@ docker exec awf-squid awk 'NF != 10' /var/log/squid/access.log ## See Also -- [LOGGING.md](../LOGGING.md) - Complete logging documentation +- [Squid Log Filtering](squid_log_filtering.md) - Filtering Squid access logs +- [Troubleshooting](troubleshooting.md) - Common issues and fixes - [README.md](../README.md) - Main project documentation diff --git a/docs/quickstart.md b/docs/quickstart.md index dda570ff..4ac2adbf 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -55,7 +55,7 @@ npm link awf --version ``` -You should see: `0.1.0` +You should see the current version number (e.g., `0.13.1`) ## Your First Command @@ -178,9 +178,9 @@ awf --allow-domains github.com \ ## Limitations ```bash -# ✗ No wildcard syntax (use base domain instead) ---allow-domains '*.github.com' ---allow-domains github.com # ✓ matches subdomains automatically +# ✓ Wildcard syntax is supported +--allow-domains '*.github.com' # matches any subdomain of github.com +--allow-domains github.com # also matches subdomains automatically # ✗ No internationalized domains (use punycode) --allow-domains bücher.ch diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 27bc8e2f..375e7a27 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -380,4 +380,4 @@ If you're still experiencing issues: - [Architecture](architecture.md) - Understand how the system works - [Usage Guide](usage.md) - Detailed usage examples - [SSL Bump](ssl-bump.md) - HTTPS content inspection and URL filtering - - [Logging Documentation](../LOGGING.md) - Comprehensive logging guide + - [Logging Quick Reference](logging_quickref.md) - Log queries and monitoring diff --git a/docs/usage.md b/docs/usage.md index a029a778..07291b5b 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -3,37 +3,45 @@ ## Command-Line Options ``` -sudo awf [options] +sudo awf [options] -- Options: --allow-domains Comma-separated list of allowed domains (optional) If not specified, all network access is blocked Example: github.com,api.github.com,arxiv.org --allow-domains-file Path to file containing allowed domains - --block-domains Comma-separated list of blocked domains - Takes precedence over allowed domains + --block-domains Comma-separated list of blocked domains + Takes precedence over allowed domains --block-domains-file Path to file containing blocked domains - --enable-host-access Enable access to host services via host.docker.internal - (see "Host Access" section for security implications) - --ssl-bump Enable SSL Bump for HTTPS content inspection - --allow-urls Comma-separated list of allowed URL patterns (requires --ssl-bump) - Example: https://github.com/myorg/*,https://api.github.com/repos/* - --log-level Log level: debug, info, warn, error (default: info) - --keep-containers Keep containers running after command exits - --work-dir Working directory for temporary files - --dns-servers Comma-separated list of DNS servers (default: 8.8.8.8,8.8.4.4) - -e, --env Additional environment variables (can repeat) - --env-all Pass all host environment variables to container - -v, --mount Volume mount (host_path:container_path[:ro|rw]) - --tty Allocate a pseudo-TTY for interactive tools - --build-local Build containers locally instead of using GHCR images - --agent-image Agent container image (default: "default") - See "Agent Image" section for available options - -V, --version Output the version number - -h, --help Display help for command + --enable-host-access Enable access to host services via host.docker.internal + (see "Host Access" section for security implications) + --allow-host-ports Ports to allow when using --enable-host-access + Example: --allow-host-ports 3000,8080 or 3000-3010 + --ssl-bump Enable SSL Bump for HTTPS content inspection + --allow-urls Comma-separated list of allowed URL patterns (requires --ssl-bump) + Example: https://github.com/myorg/* + --enable-chroot Enable chroot to /host for running host binaries + (Python, Node, Go, etc.) See chroot-mode.md + --log-level Log level: debug, info, warn, error (default: info) + --keep-containers Keep containers running after command exits + --work-dir Working directory for temporary files + --container-workdir Working directory inside the container + --dns-servers Comma-separated list of DNS servers (default: 8.8.8.8,8.8.4.4) + --proxy-logs-dir Directory to save Squid proxy logs to + -e, --env Additional environment variables (can repeat) + --env-all Pass all host environment variables to container + -v, --mount Volume mount (host_path:container_path[:ro|rw]) + --tty Allocate a pseudo-TTY for interactive tools + --build-local Build containers locally instead of using GHCR images + --agent-image Agent container image (default: "default") + See "Agent Image" section for available options + --image-registry Container image registry (default: ghcr.io/github/gh-aw-firewall) + --image-tag Container image tag (default: latest) + -V, --version Output the version number + -h, --help Display help for command Arguments: - command Command to execute (wrap in quotes) + command Command to execute (wrap in quotes, use -- separator) ``` ## Basic Examples @@ -467,6 +475,22 @@ For complete tool listings with versions, see [Agent Image Tools Reference](/gh- - Subsequent builds use Docker cache and are faster - The `full-XX.XX` images require significant disk space (~60GB extracted) +## Chroot Mode + +The `--enable-chroot` flag enables transparent access to host binaries (Python, Node.js, Go, etc.) while maintaining network isolation. This is useful for GitHub Actions runners with pre-installed tools. + +```bash +# Run with chroot mode to use host binaries +sudo awf --enable-chroot --allow-domains api.github.com \ + -- python3 -c "import requests; print(requests.get('https://api.github.com').status_code)" + +# Combine with --env-all for environment variables +sudo awf --enable-chroot --env-all --allow-domains api.github.com \ + -- bash -c 'echo "Home: $HOME, User: $USER"' +``` + +For detailed documentation including security considerations, volume mounts, and troubleshooting, see [Chroot Mode](chroot-mode.md). + ## Limitations ### No Internationalized Domains