Skip to content

Commit

Permalink
pgwire: migrate the conn/auth log to notable events
Browse files Browse the repository at this point in the history
Release note (backward-incompatible change): The connection and
authentication logging enabled by the cluster settings
`server.auth_log.sql_connections.enabled` and
`server.auth_log.sql_sessions.enabled` was previously using a text
format which was hard to parse and integrate with external monitoring
tools. This has been changed to use the standard notable event
mechanism, with standardized payloads. The output format is now
structured; see its reference documentation for details about
the supported event types and payloads.
  • Loading branch information
knz committed Jan 6, 2021
1 parent cee4753 commit 0bc8c16
Show file tree
Hide file tree
Showing 16 changed files with 3,069 additions and 135 deletions.
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -1549,6 +1549,7 @@ EVENTLOG_PROTOS = \
pkg/util/log/eventpb/privilege_events.proto \
pkg/util/log/eventpb/role_events.proto \
pkg/util/log/eventpb/zone_events.proto \
pkg/util/log/eventpb/session_events.proto \
pkg/util/log/eventpb/cluster_events.proto

docs/generated/eventlog.md: pkg/util/log/eventpb/gen.go $(EVENTLOG_PROTOS) | bin/.go_protobuf_sources
Expand Down
166 changes: 166 additions & 0 deletions docs/generated/eventlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -1183,6 +1183,172 @@ removed from a user for a type object.
| `GrantedPrivileges` | The privileges being granted to the grantee. | no |
| `RevokedPrivileges` | The privileges being revoked from the grantee. | no |

## SQL Session events

Events in this category report SQL client connections
and sessions.

They are relative to a particular SQL tenant.
In a multi-tenant setup, copies of these miscellaneous events are
preserved in each tenant's own system.eventlog table.

Events in this category are logged to channel SESSIONS.


### `client_authentication_failed`

An event of type `client_authentication_failed` is reported when a client session
did not authenticate successfully.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_sessions.enabled` is set.


| Field | Description | Sensitive |
|--|--|--|
| `Reason` | The reason for the authentication failure. | yes |
| `Detail` | The detailed error for the authentication failure. | yes |
| `Method` | The authentication method used. | yes |


#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |
| `Transport` | The connection type after transport negotiation. | yes |
| `User` | The username the session is for. This is the username passed by the client, after case-folding and Unicode normalization. | yes |

### `client_authentication_info`

An event of type `client_authentication_info` is reported for intermediate
steps during the authentication process.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_sessions.enabled` is set.


| Field | Description | Sensitive |
|--|--|--|
| `Method` | The authentication method used, once known. | yes |
| `Info` | The authentication progress message. | yes |


#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |
| `Transport` | The connection type after transport negotiation. | yes |
| `User` | The username the session is for. This is the username passed by the client, after case-folding and Unicode normalization. | yes |

### `client_authentication_ok`

An event of type `client_authentication_ok` is reported when a client session
was authenticated successfully.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_sessions.enabled` is set.


| Field | Description | Sensitive |
|--|--|--|
| `Method` | The authentication method used. | yes |


#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |
| `Transport` | The connection type after transport negotiation. | yes |
| `User` | The username the session is for. This is the username passed by the client, after case-folding and Unicode normalization. | yes |

### `client_connection_end`

An event of type `client_connection_end` is reported when a client connection
is closed. This is reported even when authentication
fails, and even for simple cancellation messages.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_connections.enabled` is set.


| Field | Description | Sensitive |
|--|--|--|
| `Duration` | The duration of the connection in nanoseconds. | no |


#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |

### `client_connection_start`

An event of type `client_connection_start` is reported when a client connection
is established. This is reported even when authentication
fails, and even for simple cancellation messages.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_connections.enabled` is set.




#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |

### `client_session_end`

An event of type `client_session_end` is reported when a client session
is completed.

Events of this type are only emitted when the cluster setting
`server.auth_log.sql_sessions.enabled` is set.


| Field | Description | Sensitive |
|--|--|--|
| `Duration` | The duration of the connection in nanoseconds. | no |


#### Common fields

| Field | Description | Sensitive |
|--|--|--|
| `Timestamp` | The timestamp of the event. Expressed as nanoseconds since the Unix epoch. | no |
| `EventType` | The type of the event. | no |
| `InstanceID` | The instance ID (not tenant ID) of the SQL server where the event was originated. | no |
| `Network` | The network protocol for this connection: tcp4, tcp6, unix, etc. | yes |
| `RemoteAddress` | The remote address of the SQL client. Note that when using a proxy or other intermediate server, this field will contain the address of the intermediate server. | yes |
| `Transport` | The connection type after transport negotiation. | yes |
| `User` | The username the session is for. This is the username passed by the client, after case-folding and Unicode normalization. | yes |

## SQL User and Role operations

Events in this category pertain to SQL statements that modify the
Expand Down
3 changes: 3 additions & 0 deletions pkg/sql/pgwire/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ go_library(
"//pkg/util/humanizeutil",
"//pkg/util/ipaddr",
"//pkg/util/log",
"//pkg/util/log/eventpb",
"//pkg/util/metric",
"//pkg/util/mon",
"//pkg/util/stop",
Expand Down Expand Up @@ -82,6 +83,7 @@ go_test(
"//pkg/sql",
"//pkg/sql/catalog/colinfo",
"//pkg/sql/parser",
"//pkg/sql/pgwire/hba",
"//pkg/sql/pgwire/pgcode",
"//pkg/sql/pgwire/pgerror",
"//pkg/sql/pgwire/pgwirebase",
Expand Down Expand Up @@ -112,6 +114,7 @@ go_test(
"@com_github_cockroachdb_datadriven//:datadriven",
"@com_github_cockroachdb_errors//:errors",
"@com_github_cockroachdb_errors//stdstrings",
"@com_github_cockroachdb_redact//:redact",
"@com_github_jackc_pgproto3_v2//:pgproto3",
"@com_github_jackc_pgx//:pgx",
"@com_github_lib_pq//:pq",
Expand Down
Loading

0 comments on commit 0bc8c16

Please sign in to comment.