[8.19](backport #44131) [libbeat/logging] fix a breaking bug

[mergify]

Proposed commit message

I think this is a breaking change that we need to address ASAP, hence tagging this PR as impact:critical.

This PR moves logging initialisation before we move forward with rest of the flow.

Issues:

1. We initialise logger at

   beats/libbeat/cmd/instance/beat.go

   Lines 1037 to 1043 in 4c1bf91

   	b.Info.Logger, err = configure.LoggingWithTypedOutputsLocal(b.Info.Beat, b.Config.Logging, b.Config.EventLogging, logp.TypeKey, logp.EventType)
   	if err != nil {
   	return fmt.Errorf("error initializing logging: %w", err)
   	}
   	// extracting here for ease of use
   	logger := b.Info.Logger

   
   But, we make use of logger in promoteOutputQueueSettings, which called is before logger's initialisation

   beats/libbeat/cmd/instance/beat.go

   Lines 1018 to 1020 in 4c1bf91

   	if err := promoteOutputQueueSettings(b); err != nil {
   	return fmt.Errorf("could not promote output queue settings: %w", err)
   	}

   
   

   beats/libbeat/cmd/instance/beat.go

   Line 1772 in 4c1bf91

   b.Info.Logger.Info("global queue settings replaced with output queue settings")

   
   Logger doesn't exist at this point and it leads to a nil dereference panic.

2. We don't really see the panic message. The command just exits without any meaningful info. We log the panic here, we make use of logp.NewLogger(...). It is a no-op logger if haven't configured the logger in the first place.

How to reproduce

Just run filebeat in normal mode with following config and you'll see nothing. No log files created, no panics printed:

filebeat.inputs:
  - type: filestream
    id: filestream-input-id
    enabled: true
    paths:
      - ./logs/file.log

output:
  elasticsearch:
    hosts: ["host"]
    username: elastic
    password: REDACTED
    bulk_max_size: 5000
    queue:
        mem:
          events: 3200
          flush:
            min_events: 1600
            timeout: 1s

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

• it fixes a disruptive user impact

  ---

  This is an automatic backport of pull request [libbeat/logging] fix a breaking bug #44131 done by Mergify.

[mergify]

Cherry-pick of 3604ffa has failed:

On branch mergify/bp/8.19/pr-44131
Your branch is up to date with 'origin/8.19'.

You are currently cherry-picking commit 3604ffadd.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   libbeat/cmd/instance/beat.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[pierrehilbert]

#44684 is required before moving on with that one.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/8.19/pr-44131 upstream/mergify/bp/8.19/pr-44131
git merge upstream/8.19
git push upstream mergify/bp/8.19/pr-44131

[khushijain21]

Closing this, will get fixed other backports

[pierrehilbert]

@khushijain21 just to be sure to understand, why did you close this PR?