📚 Documentation Noob Test Report - 2026-02-06

[github-actions]

Summary

Date of test: 2026-02-06
Test environment: GitHub Actions workflow runner (sandboxed)
Overall impression: Unable to complete testing - critical infrastructure issue prevents documentation site from running

🔴 Critical Issue Found: Documentation Build Fails in CI/Sandboxed Environments

The Problem

The documentation site cannot be built or served in GitHub Actions or sandboxed environments due to a DNS resolution failure:

[astro] Unable to load your Astro config

getaddrinfo EAI_AGAIN localhost
  Stack trace:
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:internal/dns/promises:86:17)
```

### What I Tried

1. **Build command**: `npm run build` - FAILED with EAI_AGAIN
2. **Dev server**: `npm run dev -- --port 4321 --host 127.0.0.1` - FAILED with same error
3. **Retry build**: Same failure on retry

### Impact on New Users

This is a **complete blocker** for:
- Automated documentation testing workflows
- CI/CD validation of documentation changes
- Sandboxed preview environments
- Anyone trying to build/test docs in restricted environments

### Root Cause

The Astro configuration or one of its plugins is attempting to resolve "localhost" during initialization, which fails in environments without `/etc/hosts` or DNS configured for localhost.

Likely culprits:
- `astro.config.mjs` line 36-42: Vite server configuration
- Starlight plugins that may make network calls during initialization
- Astro's dev toolbar or client prerender features

### Recommendations

#### Immediate Fix (High Priority)

1. **Add localhost DNS fallback**: Configure Astro/Vite to use `127.0.0.1` directly instead of resolving "localhost"
2. **Update Vite config**:
   ```javascript
   vite: {
       server: {
           host: '127.0.0.1', // Use IP instead of hostname
           fs: {
               allow: [
                   fileURLToPath(new URL('../', import.meta.url)),
               ],
           },
       },
   }
   ```

3. **Test in sandboxed environment**: Add a CI job that builds the docs in a minimal container to catch this issue early

#### Documentation Improvements Needed

Since I couldn't navigate the actual documentation site, here are **questions I would have as a new user** that the docs should answer clearly:

1. **What is an "agentic workflow"?**
   - Define this clearly on the home page
   - Explain how it differs from regular GitHub Actions

2. **Why would I use this instead of just writing GitHub Actions?**
   - Clear value proposition upfront
   - Show a side-by-side comparison

3. **What prerequisites do I need?**
   - List required knowledge (Git, GitHub, GitHub Actions, AI concepts)
   - List required tools (gh CLI, npm/node if needed, etc.)

4. **How much does it cost?**
   - Are there AI API costs?
   - GitHub Actions minutes usage?
   - Be transparent about costs upfront

5. **Can I try it without installing anything?**
   - Interactive demo or playground link
   - Example repository to fork and try

6. **What are "MCP servers"?**
   - This term appears in configs but needs clear explanation
   - Glossary should be linked prominently

### Next Steps

1. **Fix the localhost DNS issue** (blocks all automated testing)
2. **Add fallback documentation testing** that works in restricted environments
3. **Once fixed, re-run this noob test** to identify actual content issues

### Unable to Provide Screenshots

Due to the build failure, I was unable to:
- Take any screenshots of the documentation site
- Navigate through the getting started guide
- Identify specific confusing sections
- Test the user experience flow

**This report serves as a critical blocker** that must be resolved before meaningful documentation UX testing can occur.

---

## Test Artifacts

**Build log snippet**:
```
> docs@0.0.1 build
> astro build

[astro] Unable to load your Astro config

getaddrinfo EAI_AGAIN localhost
  Stack trace:
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:internal/dns/promises:86:17)

Environment:

• Runner OS: Linux
• Node.js version: (included in workflow)
• Working directory: /home/runner/work/gh-aw/gh-aw/docs
• /etc/hosts available: ❌ NO (file not found)

Commands attempted:

• npm install ✅ Success
• npm run build ❌ Failed (DNS error)
• npm run dev -- --port 4321 --host 127.0.0.1 ❌ Failed (DNS error)

---

Note: This was intended to be a discussion, but discussions could not be created due to permissions issues. This issue was created as a fallback.

AI generated by Documentation Noob Tester

• expires on Feb 13, 2026, 5:28 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #14287

View newer issue

---

This action was performed automatically by the Documentation Noob Tester workflow.