Bug 1862426: gather the audit logs for oauth apiserver #144

sanchezl · 2020-03-11T21:22:07Z

simplified logic
added progress logging
also capture oauth-apiserver logs
add SINCE_TIMEFRAME env, defaulted to 1w

sferich888 · 2020-07-30T12:05:10Z

@sanchezl we need to review this in the context of #143 (and what it is trying to achieve). #164 is also going to be affected by this.

sferich888

If we make these changes, we can close; #143 and work on merging this; once https://bugzilla.redhat.com/show_bug.cgi?id=1861746 is investigated and fixed.

sferich888 · 2020-07-30T12:10:07Z

collection-scripts/gather_audit_logs

+done | \
+xargs --max-args=4 --max-procs=45 bash -c \
+  'echo "INFO: Started downloading $4 from $3";
+  oc adm node-logs $3 --path=$1/$4 | gzip > $2/$3-$4.gz;


Can you add --since ${SINCE_TIMEFRAME} to this?

@sferich888 , I have added the --since parameter, but it doesn't seem to actually do anything.

The --since parameter is only applied to journal logs. It it ignored for --path logs, which are simply served by an instance of http.FileServer.

What other ways can we limit the collection?

Is an awk script needed?

#!/usr/bin/gawk -f BEGIN { starttime = mktime(starttime) endtime = mktime(endtime) } func in_range(n, start, end) { return start <= n && n < end } match($0, /^([0-9]{4})-([0-9]{2})-([0-9]{2})\s/, m) && in_range(mktime(m[1] " " m[2] " " m[3] " 00 00 00"),

Source: https://askubuntu.com/questions/800769/how-to-get-text-from-range-of-dates-using-grep-sed-in-large-text-file

@sferich888 I think I can work something out, but as a followup.

In others scripts I see the full path of command, like:/usr/bin/oc

p0lyn0mial · 2020-07-30T12:22:35Z

collection-scripts/gather_audit_logs

+# $2 - local output path
+# $3 - node name
+# $4 - log file name
+paths=(openshift-apiserver kube-apiserver)


@sanchezl please add oauth-apiserver to the list and close #164

sferich888 · 2020-07-31T16:43:40Z

@sanchezl we need to make sure we look into https://bugzilla.redhat.com/show_bug.cgi?id=1861746

sferich888 · 2020-07-31T16:45:20Z

@sanchezl also note RFE-309 is something that will be closed by this.

sanchezl · 2020-08-04T14:28:01Z

Waiting for openshift/origin#25369.

soltysh · 2020-08-11T14:47:40Z

/retets

sanchezl · 2020-08-28T02:30:25Z

/retest

soltysh · 2020-09-03T09:28:38Z

/retest

sanchezl · 2020-09-05T18:38:00Z

/test e2e-aws

sanchezl · 2020-09-06T11:41:19Z

/test-e2e-aws

openshift-ci-robot · 2020-09-06T11:43:12Z

@sanchezl: This pull request references Bugzilla bug 1862426, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.6.0) matches configured target release for branch (4.6.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1862426: gather the audit logs for oauth apiserver

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

sanchezl · 2020-09-06T11:44:02Z

/retest

soltysh · 2020-09-09T18:56:53Z

/retest

mfojtik · 2020-09-10T14:26:45Z

/approve

soltysh

/lgtm

deads2k · 2020-09-10T14:29:28Z

collection-scripts/gather

@@ -41,9 +41,6 @@ done
 # Gather etcd information
 /usr/bin/gather_etcd

-# Collect System Audit Logs
-/usr/bin/gather_audit_logs


you can't do this until the e2e test works.

that e2e you're talking about already explicitly calls this script see https://github.com/openshift/origin/blob/1a83021f4dbc91de606a71d3a27cd80664aae26c/test/extended/cli/mustgather.go#L160 so it shouldn't block this PR

deads2k · 2020-09-22T14:11:08Z

wait for openshift/origin#25493 to merge

deads2k · 2020-09-22T18:24:33Z

/test all
/lgm
/hold

need to see green tests including audit (not skipped) before merge.

sanchezl · 2020-09-23T07:06:23Z

/test all

Now that openshift/origin#25493 has merged.

sttts · 2020-09-23T08:38:16Z

/hold cancel

openshift-bot · 2020-09-23T08:45:56Z

/retest

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2020-09-23T10:29:25Z

/retest

Please review the full test history for this PR and help us cut down flakes.

- simplified logic - added progress logging - also capture oauth-apiserver logs

soltysh

Green audit e2e
/lgtm

openshift-ci-robot · 2020-09-23T14:13:59Z

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: mfojtik, sanchezl, soltysh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

collection-scripts/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci-robot · 2020-09-23T14:18:18Z

@sanchezl: All pull requests linked via external trackers have merged:

Bugzilla bug 1862426 has been moved to the MODIFIED state.

In response to this:

Bug 1862426: gather the audit logs for oauth apiserver

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Mar 11, 2020

openshift-ci-robot requested review from RickJWagner and sferich888 March 11, 2020 21:22

sferich888 suggested changes Jul 30, 2020

View reviewed changes

p0lyn0mial reviewed Jul 30, 2020

View reviewed changes

sanchezl force-pushed the patch-1 branch from 26c7582 to 3bcc9d4 Compare July 31, 2020 16:03

sferich888 mentioned this pull request Jul 31, 2020

Gather one week of node logs and drop audit logs (RFE-309) #143

Closed

sanchezl force-pushed the patch-1 branch 3 times, most recently from f1354ae to 73ce5db Compare August 2, 2020 04:30

sanchezl changed the title ~~Update gather_audit_logs~~ Bug 1862426 - gather the audit logs for oauth apiserver Aug 3, 2020

sanchezl mentioned this pull request Aug 3, 2020

Bug 1862426: gather the audit logs for oauth apiserver openshift/origin#25369

Merged

sanchezl force-pushed the patch-1 branch from 2a88c47 to d7a18d7 Compare August 4, 2020 04:02

p0lyn0mial mentioned this pull request Aug 7, 2020

workload controller for CAO openshift/cluster-authentication-operator#243

Merged

26 tasks

sferich888 mentioned this pull request Aug 24, 2020

gather_service_logs: Gather MCO host services #158

Closed

sanchezl force-pushed the patch-1 branch from d7a18d7 to 79c15df Compare September 6, 2020 04:47

sanchezl changed the title ~~Bug 1862426 - gather the audit logs for oauth apiserver~~ Bug 1862426: gather the audit logs for oauth apiserver Sep 6, 2020

openshift-ci-robot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Sep 6, 2020

soltysh approved these changes Sep 10, 2020

View reviewed changes

openshift-ci-robot assigned soltysh Sep 10, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 10, 2020

deads2k reviewed Sep 10, 2020

View reviewed changes

sanchezl force-pushed the patch-1 branch from 79c15df to 0f73b0d Compare September 22, 2020 14:04

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Sep 22, 2020

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 22, 2020

deads2k added approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. labels Sep 22, 2020

openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 23, 2020

sanchezl added 4 commits September 23, 2020 08:28

gather_audit_logs: add logging and oauth-apiserver logs

7701ea2

- simplified logic - added progress logging - also capture oauth-apiserver logs

gather_service_logs: limit service logs to one week

9ba1a64

gather: stop gathering audit logs by default

7b3fff5

gather_service_logs: fix collection & add mco services

d3bd409

sanchezl force-pushed the patch-1 branch from 0f73b0d to d3bd409 Compare September 23, 2020 12:29

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Sep 23, 2020

soltysh approved these changes Sep 23, 2020

View reviewed changes

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 23, 2020

openshift-merge-robot merged commit b35e739 into openshift:master Sep 23, 2020

sanchezl deleted the patch-1 branch January 26, 2021 20:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1862426: gather the audit logs for oauth apiserver #144

Bug 1862426: gather the audit logs for oauth apiserver #144

sanchezl commented Mar 11, 2020 •

edited

Loading

sferich888 commented Jul 30, 2020

sferich888 left a comment

sferich888 Jul 30, 2020

sanchezl Jul 31, 2020

sanchezl Jul 31, 2020

sferich888 Aug 1, 2020

sanchezl Aug 4, 2020

dougsland Aug 9, 2020

p0lyn0mial Jul 30, 2020

sanchezl Jul 31, 2020

sferich888 commented Jul 31, 2020 •

edited

Loading

sferich888 commented Jul 31, 2020

sanchezl commented Aug 4, 2020

soltysh commented Aug 11, 2020

sanchezl commented Aug 28, 2020

soltysh commented Sep 3, 2020

sanchezl commented Sep 5, 2020

sanchezl commented Sep 6, 2020

openshift-ci-robot commented Sep 6, 2020

sanchezl commented Sep 6, 2020

soltysh commented Sep 9, 2020

mfojtik commented Sep 10, 2020

soltysh left a comment

deads2k Sep 10, 2020

soltysh Sep 10, 2020

deads2k commented Sep 22, 2020

deads2k commented Sep 22, 2020

sanchezl commented Sep 23, 2020

sttts commented Sep 23, 2020

openshift-bot commented Sep 23, 2020

openshift-bot commented Sep 23, 2020

soltysh left a comment

openshift-ci-robot commented Sep 23, 2020

openshift-ci-robot commented Sep 23, 2020

Bug 1862426: gather the audit logs for oauth apiserver #144

Bug 1862426: gather the audit logs for oauth apiserver #144

Conversation

sanchezl commented Mar 11, 2020 • edited Loading

sferich888 commented Jul 30, 2020

sferich888 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sferich888 commented Jul 31, 2020 • edited Loading

sferich888 commented Jul 31, 2020

sanchezl commented Aug 4, 2020

soltysh commented Aug 11, 2020

sanchezl commented Aug 28, 2020

soltysh commented Sep 3, 2020

sanchezl commented Sep 5, 2020

sanchezl commented Sep 6, 2020

openshift-ci-robot commented Sep 6, 2020

sanchezl commented Sep 6, 2020

soltysh commented Sep 9, 2020

mfojtik commented Sep 10, 2020

soltysh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

deads2k commented Sep 22, 2020

deads2k commented Sep 22, 2020

sanchezl commented Sep 23, 2020

sttts commented Sep 23, 2020

openshift-bot commented Sep 23, 2020

openshift-bot commented Sep 23, 2020

soltysh left a comment

Choose a reason for hiding this comment

openshift-ci-robot commented Sep 23, 2020

openshift-ci-robot commented Sep 23, 2020

sanchezl commented Mar 11, 2020 •

edited

Loading

sferich888 commented Jul 31, 2020 •

edited

Loading