Env Vars Starting with USER not loading

[jhuggart]

Describe the bug
Env Vars Starting with USER not loading

To Reproduce

package config

import (
	"errors"
	"fmt"
	"github.com/knadh/koanf/providers/env"
	"github.com/knadh/koanf/v2"
	"os"
	"reflect"
	"strings"
	"testing"
)

type Config struct {
	loader *koanf.Koanf
}

var (
	ErrFailedToLoadFromEnv       = errors.New("failed to load from env")
	ErrFailedToUnmarshalToStruct = errors.New("failed to unmarshal to struct")
)

const (
	emptyPrefix = ""
	emptyDelim  = ""
	emptyPath   = ""
)

// New constructs a new config receiver
func New() *Config {
	c := &Config{
		loader: koanf.New(emptyDelim),
	}

	return c
}

// Hydrate takes a pointer to config and hydrates it
func (c *Config) Hydrate(cfgStruct any) error {
	// Load environment variables and merge into the loaded config.
	// emptyPrefix and emptyDelim. There is no env prefix and we do not need a delimiter
	if err := c.loader.Load(env.Provider(emptyPrefix, emptyDelim, func(s string) string {
		return strings.ToLower(s)
	}), nil); err != nil {
		return fmt.Errorf("%w: %v", ErrFailedToLoadFromEnv, err)
	}

	// Unmarshal to `config` tags using "FlatPaths" -> ie, don't bother with nesting config
	if err := c.loader.UnmarshalWithConf(emptyPath, cfgStruct, koanf.UnmarshalConf{
		Tag:       "config",
		FlatPaths: true,
	}); err != nil {
		return fmt.Errorf("%w: %v", ErrFailedToUnmarshalToStruct, err)
	}

	return nil
}

type TestConfig struct {
	ValueHere     string `config:"value_here"`
	UserValueHere string `config:"user_value_here"`
}

func TestHydrate(test *testing.T) {
	cases := []struct {
		name           string
		input          any
		envVals        map[string]string
		expectedOutput any
		expectedError  error
	}{
		{
			name:  "env values set correctly",
			input: TestConfig{},
			envVals: map[string]string{
				"USER_VALUE_HERE": "hi",
				"VALUE_HERE":      "hello",
			},
			expectedOutput: TestConfig{
				UserValueHere: "hi",
				ValueHere:     "hello",
			},
		},
	}

	for _, c := range cases {
		test.Run(c.name, func(t *testing.T) {
			for k, v := range c.envVals {
				if err := os.Setenv(k, v); err != nil {
					t.Fatal("failed to set env var", err)
				}
				defer os.Unsetenv(k)
			}

			conf := New()

			var toHydrate TestConfig
			if err := conf.Hydrate(&toHydrate); !errors.Is(err, c.expectedError) {
				t.Fatalf("mismatched errors - expected %v, got %v", c.expectedError, err)
			}

			if !reflect.DeepEqual(toHydrate, c.expectedOutput) {
				t.Fatalf("mismatched results - expected %v, got %v", c.expectedOutput, toHydrate)
			}
		})
	}
}

Expected behavior
Both env vars are mapped to the input struct

Please provide the following information):

• OS: mac
• Koanf Version v2.1.1

[knadh]

hmm, that can't be. It must have something to do with the environment you're in.

Works as expected on Linux.

package main

import (
	"fmt"
	"strings"

	"github.com/knadh/koanf/providers/env"
	"github.com/knadh/koanf/v2"
)

func main() {
	k := koanf.New(".")

	if err := k.Load(env.Provider("", "", func(s string) string {
		return strings.ToLower(s)
	}), nil); err != nil {
		panic(fmt.Errorf("koanf load: %v", err))
	}

	s := struct {
		ValueHere     string `config:"value_here"`
		UserValueHere string `config:"user_value_here"`
	}{}

	err := k.UnmarshalWithConf("", &s, koanf.UnmarshalConf{
		Tag:       "config",
		FlatPaths: true,
	})
	if err != nil {
		panic(err)
	}

	fmt.Println(s)
}

$ VALUE_HERE=val1 USER_VALUE_HERE=val2 go run main.go                                                                                                                                                                                    
{val1 val2}

[jhuggart]

Looks like it has something to do with the fact that I also have a USER env var on my machine. Try setting USER=anything to see if that will reproduce for you @knadh

[knadh]

$ USER=anything VALUE_HERE=val1 USER_VALUE_HERE=val2 go run main.go                                                                                                                                                                  
{val1 val2}

It's not possible that a certain env var breaks the rest of the vars. All env vars are iterated through and processed individually.

[jhuggart]

Apologies for the delayed response here. We just renamed the vars as a workaround for now. I'll work on a better reproduction next week and get back to you @knadh

[jhuggart]

If I add the following (to temporarily remove USER env var) to the test case I posted, things work as expected.

	userVal := os.Getenv("USER")
	os.Unsetenv("USER")

	defer os.Setenv("USER", userVal)

Also, if I copy-paste your code and run it exactly, I get an empty output 🤔

VALUE_HERE=hi USER_VALUE_HERE=hello go run main.go      
{ }

[jhuggart]

@knadh Can you try your code with k := koanf.New(""). I believe that's the only difference between our two implementations.

[knadh]

Ah, that's the culprit! The delim is mandatory in env.Provider(), where it does map unflattening (which splits all string keys with the given character) with the delim. If there's no delim given, it shouldn't attempt to unflatten at all. This PR fixes it: https://github.com/knadh/koanf/pull/324/files

Eg:

package main

import (
	"fmt"
	"strings"

	"github.com/knadh/koanf/providers/env"
)

func main() {
	p := env.Provider("", "", func(s string) string {
		return strings.ToLower(s)
	})

	fmt.Println(p.Read())
}

Prints a massive text dump where every character in every env key is split, ouch.

eg:
[u:map[l:map[t:map[_:map[l:map[a:map[y:map[o:map[u:map[t:us]]]]]]]]]]]]]]]]]]

[jhuggart]

🤝 Thanks for the quick PR to resolve @knadh ! When do you think that could make it into a release?

[knadh]

This is merged. You can do a go get -u on the providers/env package.