Skip to content

Improve TOML configuration error handling with position tracking and typo detection#813

Merged
lpcox merged 5 commits intomainfrom
copilot/review-burntsushi-toml-module
Feb 7, 2026
Merged

Improve TOML configuration error handling with position tracking and typo detection#813
lpcox merged 5 commits intomainfrom
copilot/review-burntsushi-toml-module

Conversation

Copy link
Contributor

Copilot AI commented Feb 7, 2026

Configuration parse errors lacked position context and silently ignored typos in field names, leading to debugging friction and silent misconfigurations.

Changes

Precise error locations

Extract line and column from ParseError.Position for pinpoint debugging:

// Before
Error: failed to parse TOML: toml: line 2 (last key "gateway"): expected '.' or '='...

// After  
Error: failed to parse TOML at line 2, column 6: expected '.' or '=', but got '3' instead

Unknown key detection

Use MetaData.Undecoded() to catch typos and deprecated fields:

[gateway]
prot = 3000              # Typo detected
startup_timout = 30      # Typo detected

Generates warnings:

[WARN] [config] Unknown configuration key 'gateway.prot' - check for typos
[WARN] [config] Unknown configuration key 'gateway.startup_timout' - check for typos

Streaming decoder

Replace os.ReadFile + toml.Decode with toml.NewDecoder for memory efficiency with large config files.

Test coverage

  • Parse errors with column numbers
  • Unknown key detection in gateway and server sections
  • Multiple simultaneous typos
  • Large file handling (100+ servers)
  • TOML 1.1 duplicate key detection

Documentation

Added "Configuration Validation and Error Handling" section to README with examples and best practices.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • example.com
    • Triggering command: /tmp/go-build4210791917/b271/launcher.test /tmp/go-build4210791917/b271/launcher.test -test.testlogfile=/tmp/go-build4210791917/b271/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/net -trimpath 64/pkg/tool/linux_amd64/vet -p github.com/davec--norc -lang=go1.16 64/pkg/tool/linux_amd64/vet -I 519236/b216/_pkg_.a cfg fig.test --gdwarf-5 b/gh-aw-mcpg/intdocker-cli-plugin-metadata -o fig.test (dns block)
    • Triggering command: `/tmp/go-build3777671178/b275/launcher.test /tmp/go-build3777671178/b275/launcher.test -test.testlogfile=/tmp/go-build3777671178/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true t n for TOML configuration validation and error handling

Co-autho--format=format:%H %ct %D ache/go/1.25.6/x64/pkg/tool/linux_amd64/vet ache/go/1.25.6/x/opt/hostedtoolcache/go/1.25.6/x64/pkg/tool/linux_amd64/compile -I` (dns block)

  • invalid-host-that-does-not-exist-12345.com
    • Triggering command: /tmp/go-build3314539425/b001/config.test /tmp/go-build3314539425/b001/config.test -test.testlogfile=/tmp/go-build3314539425/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true go k1BRF76vb x_amd64/vet credential.helpe/opt/hostedtoolcache/go/1.25.6/x64/pkg/tool/linux_amd64/compile (dns block)
    • Triggering command: /tmp/go-build4210791917/b259/config.test /tmp/go-build4210791917/b259/config.test -test.testlogfile=/tmp/go-build4210791917/b259/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/net ache/go/1.25.6/x64/src/go/scanner/errors.go bin/as -I /tmp/go-build962-o -I 1.o -o 64/src/net -trimpath 64/pkg/tool/linu-lang=go1.25 -p crypto/sha256 -lang=go1.25 64/pkg/tool/linu-goversion (dns block)
    • Triggering command: `/tmp/go-build3777671178/b263/config.test /tmp/go-build3777671178/b263/config.test -test.testlogfile=/tmp/go-build3777671178/b263/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true .cfg by/8ae1b12e94aa80f9eaed17e6ee8f6f9b1e1c0807ed03bea0920b88cdb129e6eb 64/pkg/tool/linux_amd64/vet by/8ae1b12e94aa8git 920b88cdb129e6ebrev-parse rgo/bin/as 64/pkg/tool/linux_amd64/vet dock�� -o n for TOML configuration validation and error handling

Co-autho-buildid=HBV11mEKIlvfULJyOluM/u0/tmp/go-build673569813/b262/vet.cfg ache/go/1.25.6/x64/pkg/tool/linux_amd64/vet ache/go/1.25.6/xbash -I ache/go/1.25.6/x--noprofile ache/go/1.25.6/x64/pkg/tool/linux_amd64/vet` (dns block)

  • nonexistent.local
    • Triggering command: /tmp/go-build4210791917/b271/launcher.test /tmp/go-build4210791917/b271/launcher.test -test.testlogfile=/tmp/go-build4210791917/b271/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/net -trimpath 64/pkg/tool/linux_amd64/vet -p github.com/davec--norc -lang=go1.16 64/pkg/tool/linux_amd64/vet -I 519236/b216/_pkg_.a cfg fig.test --gdwarf-5 b/gh-aw-mcpg/intdocker-cli-plugin-metadata -o fig.test (dns block)
    • Triggering command: `/tmp/go-build3777671178/b275/launcher.test /tmp/go-build3777671178/b275/launcher.test -test.testlogfile=/tmp/go-build3777671178/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true t n for TOML configuration validation and error handling

Co-autho--format=format:%H %ct %D ache/go/1.25.6/x64/pkg/tool/linux_amd64/vet ache/go/1.25.6/x/opt/hostedtoolcache/go/1.25.6/x64/pkg/tool/linux_amd64/compile -I` (dns block)

  • slow.example.com
    • Triggering command: /tmp/go-build4210791917/b271/launcher.test /tmp/go-build4210791917/b271/launcher.test -test.testlogfile=/tmp/go-build4210791917/b271/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true 64/src/net -trimpath 64/pkg/tool/linux_amd64/vet -p github.com/davec--norc -lang=go1.16 64/pkg/tool/linux_amd64/vet -I 519236/b216/_pkg_.a cfg fig.test --gdwarf-5 b/gh-aw-mcpg/intdocker-cli-plugin-metadata -o fig.test (dns block)
    • Triggering command: `/tmp/go-build3777671178/b275/launcher.test /tmp/go-build3777671178/b275/launcher.test -test.testlogfile=/tmp/go-build3777671178/b275/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true t n for TOML configuration validation and error handling

Co-autho--format=format:%H %ct %D ache/go/1.25.6/x64/pkg/tool/linux_amd64/vet ache/go/1.25.6/x/opt/hostedtoolcache/go/1.25.6/x64/pkg/tool/linux_amd64/compile -I` (dns block)

  • this-host-does-not-exist-12345.com
    • Triggering command: /tmp/go-build4210791917/b280/mcp.test /tmp/go-build4210791917/b280/mcp.test -test.testlogfile=/tmp/go-build4210791917/b280/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true /opt/hostedtoolcache/go/1.25.6/x64/src/runtime/cgo 519236/b184/ 64/bin/go --gdwarf-5 --64 ut-2370224496.c as 5192�� ache/go/1.25.6/x64/src/net -I cfg --gdwarf-5 --64 -o RL1q3u8/2td8KcPQCBPBYpO4RxDG (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[go-fan] Go Module Review: BurntSushi/toml</issue_title>
<issue_description># 🐹 Go Fan Report: github.com/BurntSushi/toml

Module Overview

BurntSushi/toml is the de-facto standard TOML parser for Go with 4,893 stars and active maintenance. This library provides a reflection-based API for encoding and decoding TOML configuration files, implementing the TOML specification with excellent compliance. The project is currently using v1.6.0 (released December 18, 2025).

Current Usage in gh-aw-mcpg

The TOML library has a focused, minimal footprint in the gateway:

  • Files: 1 file (internal/config/config_core.go)
  • Import Count: 1 import
  • Key APIs Used:
    • toml.Decode() - Single usage for parsing TOML configuration into Go structs

Current implementation (lines 97-98 of config_core.go):

if _, err := toml.Decode(string(data), &cfg); err != nil {
    return nil, fmt.Errorf("failed to parse TOML: %w", err)
}
``````

### Usage Pattern Analysis
✅ **Strengths**: Clean, simple, effective for basic parsing  
⚠️ **Limitations**: 
- Generic error wrapping loses valuable position information
- Reads entire file into memory (inefficient for large configs)
- No validation of unknown/misspelled keys
- Doesn't leverage advanced features added in recent versions

## Research Findings

### Recent Updates (We're on Latest! 🎉)
The project is already using v1.6.0, which includes:

**v1.6.0 (December 2025)** - Current Version:
- ✅ **TOML 1.1 enabled by default** - Allows newlines in inline tables for better readability
- ✅ **Fixed duplicate array key validation** - Catches configuration errors like:
  ``````toml
  arr = [1]
  arr = [2]  # Now correctly errors: "Key 'arr' has already been defined"
  ``````
- ✅ **Improved float encoding** - Large floats now use exponent syntax (e.g., `5e+22`)

**v1.5.0 (March 2025)** - Available features we're not using:
- 📍 **`Position.Col`** - Column-level error reporting for pinpoint accuracy
- 🔑 **Custom string types as map keys** - More flexible type handling
- 🐛 **Better nested inline table encoding** - Fixes edge cases

**v1.4.0 (May 2024)** - Available features we're not using:
- 📦 **`toml.Marshal()`** - Convenient encoding API for generating TOML
- 🎯 **Position wrapping** - Better error context from `UnmarshalTOML()`

### Best Practices from Maintainers

1. **Leverage ParseError for detailed diagnostics**
   ``````go
   if perr, ok := err.(*toml.ParseError); ok {
       // Access: perr.Line, perr.Col, perr.Message
   }
   ``````

2. **Use Meta API to detect typos/unknown keys**
   ``````go
   md, err := toml.Decode(string(data), &cfg)
   for _, key := range md.Undecoded() {
       log.Printf("WARNING: Unknown key: %s", key)
   }
   ``````

3. **Stream large files with NewDecoder**
   ``````go
   decoder := toml.NewDecoder(file)
   decoder.Strict(true)  // Reject unknown fields
   ``````

4. **Implement UnmarshalTOML for custom validation**
   - Validate fields during unmarshaling
   - Return descriptive errors with field names

## Improvement Opportunities

### 🏃 Quick Wins

#### 1. **Leverage Position Information for Better Error Messages** (5 minutes)
**Impact**: High - Users get actionable error messages  
**Effort**: Minimal - Simple type assertion

**Current problem**: 
``````
Error: failed to parse TOML: near line 15: expected key separator '='

Improved version:

if _, err := toml.Decode(string(data), &cfg); err != nil {
    if perr, ok := err.(*toml.ParseError); ok {
        return nil, fmt.Errorf("failed to parse TOML at line %d, column %d: %s", 
            perr.Line, perr.Col, perr.Message)
    }
    return nil, fmt.Errorf("failed to parse TOML: %w", err)
}
``````

**Result**:
``````
Error: failed to parse TOML at line 15, column 8: expected key separator '='
                                      ^
                                    Exact location!

2. Detect Configuration Typos with Meta API (10 minutes)

Impact: High - Catch common mistakes like prot instead of port
Effort: Low - Add 5 lines of code

var cfg Config
md, err := toml.Decode(string(data), &cfg)
if err != nil {
    return nil, err
}

// Warn about unknown keys (typos, deprecated options)
undecoded := md.Undecoded()
if len(undecoded) > 0 {
    for _, key := range undecoded {
        logConfig.Printf("WARNING: Unknown configuration key '%s' - check for typos", key)
    }
}

Catches errors like:

  • prot = 3000 → warns about prot, suggests port
  • startup_timout = 60 → warns about startup_timout, suggests startup_timeout

3. Use Streaming Decoder (10 minutes)

Impact: Medium - Better memory efficiency
Effort: Low - Replace file read with decoder

// Before: Load entire file
data, err := os.ReadFile(path)
if _, err := toml.Decode(string(data), &cfg); err != nil { ... }

// After: Strea...

</details>


> **Custom agent used: agentic-workflows**
> GitHub Agentic Workflows (gh-aw) - Create, debug, and upgrade AI-powered workflows with intelligent prompt routing



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes github/gh-aw-mcpg#749

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).

Copilot AI and others added 4 commits February 7, 2026 17:47
…o detection

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Review BurntSushi/toml Go module usage Improve TOML configuration error handling with position tracking and typo detection Feb 7, 2026
Copilot AI requested a review from lpcox February 7, 2026 17:53
@lpcox lpcox marked this pull request as ready for review February 7, 2026 18:00
Copilot AI review requested due to automatic review settings February 7, 2026 18:00
@lpcox lpcox merged commit a7b390d into main Feb 7, 2026
5 checks passed
@lpcox lpcox deleted the copilot/review-burntsushi-toml-module branch February 7, 2026 18:00
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves TOML configuration diagnostics in the gateway by adding precise parse error locations (line/column), warning on unknown/typoed keys, and switching to streaming TOML decoding for better memory behavior.

Changes:

  • Parse TOML via toml.NewDecoder and wrap toml.ParseError with line/column details.
  • Detect unknown TOML keys via MetaData.Undecoded() and emit warnings.
  • Add tests and update documentation/examples describing the new behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
internal/config/config_core.go Switches to streaming TOML decode; adds line/column parse errors and unknown-key warnings via logger.
internal/config/config_test.go Adds tests covering parse errors with column info, unknown key scenarios, large config parsing, and duplicate keys.
config.example.toml Updates example comments to describe unknown-key warnings and line/column parse errors.
README.md Adds a new section explaining config validation/error handling with examples.
Comments suppressed due to low confidence (3)

internal/config/config_test.go:1192

  • TestLoadFromFile_MultipleUnknownKeys similarly doesn't assert that multiple unknown keys are actually reported; it only checks that misspelled fields fell back to defaults. To ensure the behavior described in the test name, capture the warning output and assert it includes each expected unknown key (e.g., gateway.startup_timout, gateway.tool_timout, servers.test.typ).
	// Enable debug logging to capture warnings
	SetDebug(true)
	defer SetDebug(false)

	// Should still load successfully
	cfg, err := LoadFromFile(tmpFile)
	require.NoError(t, err, "LoadFromFile() should succeed even with multiple unknown keys")
	require.NotNil(t, cfg, "Config should not be nil")

	// Correctly spelled fields should work
	assert.Equal(t, 8080, cfg.Gateway.Port, "Port should be set correctly")
	// Misspelled fields should use defaults
	assert.Equal(t, DefaultStartupTimeout, cfg.Gateway.StartupTimeout, "StartupTimeout should be default")
	assert.Equal(t, DefaultToolTimeout, cfg.Gateway.ToolTimeout, "ToolTimeout should be default")

internal/config/config_test.go:1245

  • In TestLoadFromFile_InvalidTOMLDuplicateKey, the comment says the error should mention the duplicate key, but the assertion only checks for "line". This makes the test less meaningful and could miss regressions where a different parse error happens. Consider asserting the error mentions the duplicated key name (e.g., "port") or a duplicate/"already defined" message in addition to the position info.
	// Error should mention the duplicate key
	assert.Contains(t, err.Error(), "line", "Error should mention line number")
}

internal/config/config_test.go:1122

  • TestLoadFromFile_ParseErrorWithColumnNumber allows passing when the error only contains "line 2" but not "column". Since the test name/PR intent is to ensure column reporting, this should assert the presence of the "column" token (or a regex like line \d+, column \d+) so it actually fails if column info regresses.
	// Error should contain line and column information from our improved error format
	errMsg := err.Error()
	assert.Contains(t, errMsg, "line", "Error should mention line number")
	// Our improved format includes "column" explicitly when ParseError is detected
	assert.True(t, strings.Contains(errMsg, "column") || strings.Contains(errMsg, "line 2"),
		"Error should mention column or line position, got: %s", errMsg)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +1144 to +1155
// Enable debug logging to capture warning about unknown key
SetDebug(true)
defer SetDebug(false)

// Should still load successfully, but warning will be logged
cfg, err := LoadFromFile(tmpFile)
require.NoError(t, err, "LoadFromFile() should succeed even with unknown keys")
require.NotNil(t, cfg, "Config should not be nil")

// Port should be default since "prot" was not recognized
assert.Equal(t, DefaultPort, cfg.Gateway.Port, "Port should be default since 'prot' is unknown")
assert.Equal(t, "test-key", cfg.Gateway.APIKey, "API key should be set correctly")
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestLoadFromFile_UnknownKeysInGateway claims to test unknown-key detection, but it never asserts that a warning was emitted (it only checks defaults). This can pass even if the warning/logging path regresses. Consider capturing stderr output from SetDebug(true) (or initializing the file logger in-test) and asserting the log contains the expected "Unknown configuration key" message and the specific key name.

This issue also appears in the following locations of the same file:

  • line 1179
  • line 1243
  • line 1117

Copilot uses AI. Check for mistakes.
return nil, fmt.Errorf("failed to parse TOML at line %d, column %d: %s",
perr.Position.Line, perr.Position.Col, perr.Message)
}
// Try value type (used by toml.Decode)
Copy link

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment "Try value type (used by toml.Decode)" is misleading now that this function uses toml.NewDecoder(file).Decode. If the value-type assertion is kept for compatibility, consider updating the comment to describe when/why toml.ParseError might be returned as a value in this code path.

Suggested change
// Try value type (used by toml.Decode)
// Also handle value-type ParseError for compatibility with older or alternate error paths

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants