fix provider config file not used in library mode

[nth347]

Proposed Changes

• fix provider config file not loaded in library mode
• adds updated example usage in library mode at v2/examples/main.go
• run example as part of build test workflow
• closes provider config not used/loaded in library mode #854 add example file for library usage #867

Refer to the issue #854

The problem is Subfinder unable to load provider-config.yaml file. The pull request added ParseUserOptions function to provide library users to parse their own Options. Example:

func main() {
	// Parse the command line flags and read config files
	options := &runner.Options{
		ProviderConfig: "C:\\Users\\nth347\\.config\\subfinder\\provider-config.yaml", // Does not take effect if you don't use ParseUserOptions
		Output:         os.Stdout,
		Domain:         []string{"target.com"},
	}
	//options.loadProvidersFrom("C:\\Users\\nth347\\.config\\subfinder\\provider-config.yaml")
	parsedOptions := runner.ParseUserOptions(options)

	newRunner, err := runner.NewRunner(parsedOptions)
	if err != nil {
		gologger.Fatal().Msgf("Could not create runner: %s\n", err)
	}

	err = newRunner.RunEnumeration()
	if err != nil {
		gologger.Fatal().Msgf("Could not run enumeration: %s\n", err)
	}
}

[nth347]

Hi @ehsandeep , please review my pull request

[nth347]

Hi @tarunKoyalwar and @dogancanbakir,

To allow library users to load their own provider-config.yaml file, we need to make the func (options *Options) loadProvidersFrom(location string) function public. However, currently, Subfinder does not have an API for loading config.yaml. In order to address this, I have decided to introduce a new function called func ParseUserOptions(options *Options) *Options. This new function performs nearly the same operations as the existing func ParseOptions() *Options function, with the difference being that it returns the Options object for further usage & customization by library users.

[dogancanbakir]

Thanks for the PR, @nth347. I just left some comments for you.

[nth347]

Hi @dogancanbakir after spending some time debugging, it turns out that the new Init function is essentially the same as the ParseOptions function. However, I did make a change on line 406. If library users specify their config.yaml location, it would be beneficial to merge it into the current Options. For now, library users only need to do something like the following:

func main() {
	options := &runner.Options{
		ResultCallback: func(s *resolve.HostEntry) { // Callback function to execute for available host
			log.Println(s.Host, s.Source)
		},
		ProviderConfig: "/path/to/provider-config.yaml", // Will take effect if you call runner.Init
		Config:         "/path/to/config.yaml", // Will take effect if you call runner.Init
		Domain:         []string{"tesla.com"},
	}

	parsedOptions := runner.Init(options)

	newRunner, err := runner.NewRunner(parsedOptions)
	if err != nil {
		gologger.Fatal().Msgf("Could not create runner: %s\n", err)
	}

	err = newRunner.RunEnumeration()
	if err != nil {
		gologger.Fatal().Msgf("Could not run enumeration: %s\n", err)
	}
}

[tarunKoyalwar]

@nth347 thanks for flagging the issue and creating PR to fix that

I simplified the logic so there is no need to invoke another function like Init() and the updated example for using subfinder as library is available at https://github.com/nth347/subfinder/blob/02f520a0b1926224b8fbf57501a8112d1a7112ee/v2/examples/main.go#L12-L50

[nth347]

Hi @tarunKoyalwar, thank you for your fix! I noticed the problem, but I'm still working on improving my coding skills to address it. Your solution is greatly appreciated, and you did an excellent job!

[tarunKoyalwar]

@nth347 , no worries also don't hesitate to raise PR thinking about coding skills / being up to the mark . pd-team will help you with PRs as much as we can , just create a draft PR and tag anyone from pdteam