[Winlogbeat] ECS 1.9 user.changes., user.effective., user.target.* #26509

janniten · 2021-06-27T18:06:16Z

What does this PR do?

In ECS 1.9 user.changes.*, user.effective.*, and user.target.* were introduced in order to capture better those events in where many users are involved. This fields allows us to model complex user's relationships.

See improvements sections in https://github.com/elastic/ecs/releases

Why is it important?

According to the usage described in https://www.elastic.co/guide/en/ecs/current/ecs-user-usage.html modifications to the winlogbeat security module are introduced in this PR in order to model user's relationship in an event.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

Use cases

The events affected are

EventID	User Field	Comments
4648	user.effective.*	This event captures the semantic of RunAs. Originally the user.* was completed with the information of the winlog.event_data.TargetUser., but according to the documentation user.* is the actual user who is executing the RunAs,* so I've change to the user.* to be complete with the winlog.event_data.SubjectUser and the winlog.event_data.TargetUser.* copied to user.effective.. From the doc Use the user fields at the root to capture who is requesting the privilege change, and user.effective to capture the requested privilege level, whether or not the privilege change was successful*
4688	user.effective.*	In Windows 10/ Windows Server 2016 the some fields (winlog.event_data.TargetUser.*) where added in order to capture when a process is started under a different account. By default, a new process runs under the same account and logon session as the creator process
4720	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4722	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4723	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4724	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4725	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4726	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4728	user.target.*	Group Management Events user.target.* is completed with member information
4729	user.target.*	Group Management Events user.target.* is completed with member information
4732	user.target.*	Group Management Events user.target.* is completed with member information
4733	user.target.*	Group Management Events user.target.* is completed with member information
4738	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4740	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4746	user.target.*	Group Management Events user.target.* is completed with member information
4747	user.target.*	Group Management Events user.target.* is completed with member information
4751	user.target.*	Group Management Events user.target.* is completed with member information
4752	user.target.*	Group Management Events user.target.* is completed with member information
4756	user.target.*	Group Management Events user.target.* is completed with member information
4757	user.target.*	Group Management Events user.target.* is completed with member information
4761	user.target.*	Group Management Events user.target.* is completed with member information
4762	user.target.*	Group Management Events user.target.* is completed with member information
4767	user.target.*	User Management Events where the winlog.event_data.TargetUser.* -> user.target.*
4768	fix	Fixed targetSid Field
4771	fix	Fixed targetSid Field
4781	user.target.* and user.changes.*	User Management Events where the winlog.event_data.OldTargetUser.* -> user.target.* and winlog.event_data.NewTargetUser.*

elasticmachine · 2021-06-27T18:10:38Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2021-10-18T11:54:47.349+0000
Duration: 62 min 27 sec
Commit: 8941a65

Test stats 🧪

Test	Results
Failed	0
Passed	880
Skipped	0
Total	880

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.

janniten · 2021-06-27T18:11:02Z

@jsoriano #25754

elasticmachine · 2021-06-28T09:57:51Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

jsoriano · 2021-06-28T09:58:22Z

/test

x-pack/winlogbeat/module/security/config/winlogbeat-security.js

mergify · 2021-07-12T20:55:47Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b ecs_1.9 upstream/ecs_1.9
git merge upstream/master
git push upstream ecs_1.9

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>

janniten · 2021-08-23T12:54:59Z

Hi @adriansr @jsoriano
I've been using the latest version of winlogbeat and I realize that for some events (4728,4732,4733,4729) the field user.target.* is been populated.
In this case I've noticed that the user.target.name is populated with the group name, and not with the user that is been added to the group.
IMHO, I think that the user.target.name should be the user added to the group because is affecting the user's properties

        evt.Put("user.target.group.id", evt.Get("group.id"));
        evt.Put("user.target.group.name", evt.Get("group.name"));
        evt.Put("user.target.group.domain", evt.Get("group.domain"));

Also the user.target.* user.effective.* and user.changes.* are part of the winlogbeat's index template but not completed for the User Management Events and or 4648 and 4688
Shall I cancel this PR? and open a new one with the events still not have the user.target/effective/changes ?
Or just cancel it definitively if you are working in those changes.
Thank you
Regards
Ana

adriansr · 2021-09-09T14:48:58Z

Hi @janniten we're looking at this, sorry for the delay

mergify · 2021-09-22T12:54:35Z

This pull request does not have a backport label. Could you fix it @janniten? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

P1llus · 2021-10-14T08:04:12Z

/test

marc-gr · 2021-10-14T10:46:57Z

/test

marc-gr · 2021-10-14T11:36:46Z

...eat/module/security/test/testdata/security-windows2019_4688_Process_Created.evtx.golden.json

 },
 "user": {
 "domain": "VAGRANT",
+ "effective": {
+ "domain": "-",


would be nice if we could ignore -

marc-gr · 2021-10-14T11:38:19Z

CHANGELOG.next.asciidoc

@@ -439,6 +439,8 @@ for a few releases. Please use other tools provided by Elastic to fetch data fro
 - Add source.ip validation for event ID 4778 in the Security module. {issue}19627[19627]
 - Protect against accessing undefined variables in Sysmon module. {issue}22219[22219] {pull}22236[22236]
 - Protect against accessing an undefined variable in Security module. {pull}22937[22937]
+- Fix related.ip field in renameCommonAuthFields {pull}24892[24892]


I think this line got in with the merge, can we remove it if it is the case?

…kages/cisco_meraki values

marc-gr · 2021-10-18T08:20:58Z

/test

marc-gr · 2021-10-18T11:54:16Z

/test

P1llus

LGTM

changes done

…26509) * Add Winlogbeat Security Module Doc * ECS 1.9 new user fields * Add Documentation * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Suggeted changes by adriansr * Regenerate golden files * Fix changelog and remove ~/go/src/github.com/elastic/integrations/packages/cisco_meraki values * Fix typo * Regenerate test files * Check for empty values on target user Co-authored-by: Adrian Serrano <adrisr83@gmail.com> Co-authored-by: Marc Guasch <marc.guasch@elastic.co> (cherry picked from commit 04ce8a5)

…26509) (#28525) * Add Winlogbeat Security Module Doc * ECS 1.9 new user fields * Add Documentation * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Suggeted changes by adriansr * Regenerate golden files * Fix changelog and remove ~/go/src/github.com/elastic/integrations/packages/cisco_meraki values * Fix typo * Regenerate test files * Check for empty values on target user Co-authored-by: Adrian Serrano <adrisr83@gmail.com> Co-authored-by: Marc Guasch <marc.guasch@elastic.co> (cherry picked from commit 04ce8a5) Co-authored-by: Anabella Cristaldi <33020901+janniten@users.noreply.github.com>

…lastic#26509) * Add Winlogbeat Security Module Doc * ECS 1.9 new user fields * Add Documentation * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js Co-authored-by: Adrian Serrano <adrisr83@gmail.com> * Suggeted changes by adriansr * Regenerate golden files * Fix changelog and remove ~/go/src/github.com/elastic/integrations/packages/cisco_meraki values * Fix typo * Regenerate test files * Check for empty values on target user Co-authored-by: Adrian Serrano <adrisr83@gmail.com> Co-authored-by: Marc Guasch <marc.guasch@elastic.co>

janniten and others added 3 commits January 26, 2021 09:38

Add Winlogbeat Security Module Doc

71d916b

Merge branch 'elastic:master' into master

2a610c4

ECS 1.9 new user fields

8a5db71

janniten requested a review from a team as a code owner June 27, 2021 18:06

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 27, 2021

Add Documentation

f7acd39

janniten mentioned this pull request Jun 27, 2021

ECS 1.9 user.changes.*, user.effective.*, user.target.* #25754

Closed

5 tasks

jsoriano added the Team:Security-External Integrations label Jun 28, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 28, 2021

adriansr changed the title ECS 1.9 user.changes.*, user.effective.*, user.target.* [Winlogbeat] ECS 1.9 user.changes.*, user.effective.*, user.target.* Jun 28, 2021

adriansr added ecs enhancement review Winlogbeat labels Jun 28, 2021

adriansr previously requested changes Jul 12, 2021

View reviewed changes

janniten and others added 3 commits July 17, 2021 20:43

Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js

4788f66

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>

Update x-pack/winlogbeat/module/security/config/winlogbeat-security.js

6e84caf

Co-authored-by: Adrian Serrano <adrisr83@gmail.com>

Suggeted changes by adriansr

2167b12

janniten requested a review from adriansr July 20, 2021 16:05

adriansr requested a review from marc-gr September 9, 2021 14:58

marc-gr added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label Sep 10, 2021

mergify bot added the backport-skip Skip notification from the automated backport with mergify label Sep 22, 2021

marc-gr self-assigned this Sep 27, 2021

janniten and others added 2 commits October 13, 2021 19:08

Merge branch 'elastic:master' into master

0727ab0

Merge remote-tracking branch 'upstream/master' into ecs_1.9

4bd4bca

Regenerate golden files

3c67931

marc-gr reviewed Oct 14, 2021

View reviewed changes

marc-gr added 3 commits October 18, 2021 10:15

Fix changelog and remove ~/go/src/github.com/elastic/integrations/pac…

6a94aa4

…kages/cisco_meraki values

Fix typo

8d3c800

Regenerate test files

782721b

Check for empty values on target user

8941a65

P1llus approved these changes Oct 19, 2021

View reviewed changes

marc-gr merged commit 04ce8a5 into elastic:master Oct 19, 2021

marc-gr added backport-v7.16.0 Automated backport with mergify and removed backport-skip Skip notification from the automated backport with mergify labels Oct 19, 2021

mergify bot mentioned this pull request Oct 19, 2021

[7.x](backport #26509) [Winlogbeat] ECS 1.9 user.changes.*, user.effective.*, user.target.* #28525

Merged

janniten deleted the ecs_1.9 branch October 19, 2021 08:33

marc-gr mentioned this pull request Oct 19, 2021

[system] Improve user mappings in security elastic/integrations#1944

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Winlogbeat] ECS 1.9 user.changes., user.effective., user.target.* #26509

[Winlogbeat] ECS 1.9 user.changes., user.effective., user.target.* #26509

janniten commented Jun 27, 2021 •

edited

Loading

elasticmachine commented Jun 27, 2021 •

edited by jenkins-beats-ci bot

Loading

Build stats

Test stats 🧪

janniten commented Jun 27, 2021

elasticmachine commented Jun 28, 2021

jsoriano commented Jun 28, 2021

mergify bot commented Jul 12, 2021

janniten commented Aug 23, 2021

adriansr commented Sep 9, 2021

mergify bot commented Sep 22, 2021

P1llus commented Oct 14, 2021

marc-gr commented Oct 14, 2021

marc-gr Oct 14, 2021

marc-gr Oct 14, 2021

marc-gr commented Oct 18, 2021

marc-gr commented Oct 18, 2021

P1llus left a comment

[Winlogbeat] ECS 1.9 user.changes.*, user.effective.*, user.target.* #26509

[Winlogbeat] ECS 1.9 user.changes.*, user.effective.*, user.target.* #26509

Conversation

janniten commented Jun 27, 2021 • edited Loading

What does this PR do?

Why is it important?

Checklist

Author's Checklist

Use cases

elasticmachine commented Jun 27, 2021 • edited by jenkins-beats-ci bot Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

janniten commented Jun 27, 2021

elasticmachine commented Jun 28, 2021

jsoriano commented Jun 28, 2021

mergify bot commented Jul 12, 2021

janniten commented Aug 23, 2021

adriansr commented Sep 9, 2021

mergify bot commented Sep 22, 2021

P1llus commented Oct 14, 2021

marc-gr commented Oct 14, 2021

marc-gr Oct 14, 2021

Choose a reason for hiding this comment

marc-gr Oct 14, 2021

Choose a reason for hiding this comment

marc-gr commented Oct 18, 2021

marc-gr commented Oct 18, 2021

P1llus left a comment

Choose a reason for hiding this comment

[Winlogbeat] ECS 1.9 user.changes., user.effective., user.target.* #26509

[Winlogbeat] ECS 1.9 user.changes., user.effective., user.target.* #26509

janniten commented Jun 27, 2021 •

edited

Loading

elasticmachine commented Jun 27, 2021 •

edited by jenkins-beats-ci bot

Loading