[winlogbeat] message ids not expanded in individual fields

[jdu2600]

Windows Event logs can include localised strings. To achieve this the raw event includes message ids that need to be looked up in localisation message tables.

Currently winlogbeat does this localisation for the message field, but not for the original individual fields.

Extracts showing the behaviour here -

{
  "winlog.event_data": {
    "VirtualAccount": "%%1843",
    "ElevatedToken": "%%1842",
    "ImpersonationLevel": "%%1833",
  },
  "message": "...Virtual Account:\t\tNo\n\tElevated Token:\t\tYes\n\nImpersonation Level:\t\tImpersonation...",
}

{
  "winlog.event_data": {
    "FailureReason": "%%2304",
  },
  "message": "...Failure Reason:\t\tAn Error occured during Logon.\n...",
}

The message id can be resolved using kernel32!FormatMessage.

[elasticmachine]

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

[intxgo]

does this concerns events originating from localhost or forwarded events? In the latter case, winlogbeat doesn't attempt doing any substitutions on it's own as the resource dlls most likely are not present on the WEC server. Forwarded events are just trivially converted from xml to json.

[jdu2600]

I suspect both - but have not confirmed.
iirc the XML includes the message ids and it is the responsibility of the client (eg Event Viewer, winlogbeat) to localise them.

For System events the resource dll will be on the WEC server - so we could make a best effort attempt.
At a glance it looks like we do this currently but only for the message field, and the other event data fields are left as is.

beats/winlogbeat/sys/wineventlog/renderer.go

Lines 121 to 126 in 23bbe8c

	// Associate key names with the event data values.
	r.addEventData(eventMeta, eventData, event)
	if event.Message, err = r.formatMessage(md, eventMeta, handle, eventData, uint16(event.EventIdentifier.ID)); err != nil {
	errs = append(errs, fmt.Errorf("failed to get the event message string: %w", err))
	}

[marc-gr]

For System events the resource dll will be on the WEC server - so we could make a best effort attempt.

Is this always true? We could do a best effort as you mention and stop trying if it fails.

[jdu2600]

Is this always true?

If it wasn't true then our message field would also have the raw %% message ids, right?

During addEventData we should be able to call kernel32!FormatMessage (or wevtapi!EvtFormatMessage) if the extracted value matches ^%%[0-9]+$.

[marc-gr]

Oh I think I misunderstood when you meant System as System events even if they were forwarded. My bad.