Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support to audit the authentication http request #3951

Closed
wants to merge 10 commits into from
Closed

Support to audit the authentication http request #3951

wants to merge 10 commits into from

Conversation

turboFei
Copy link
Member

@turboFei turboFei commented Dec 8, 2022 •
edited
Loading

Why are the changes needed?

Support to audit the http request.

08:10:43.231 INFO AuthenticationAuditLogger: user=fwang12(auth:BASIC)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=200
08:10:43.265 INFO AuthenticationAuditLogger: user=null(auth:BASIC)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=403
08:10:43.273 INFO AuthenticationAuditLogger: user=null(auth:null)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=401
08:10:43.320 INFO AuthenticationAuditLogger: user=client(auth:NEGOTIATE)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=200
08:10:43.324 INFO AuthenticationAuditLogger: user=null(auth:NEGOTIATE)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=403
08:10:43.331 INFO AuthenticationAuditLogger: user=null(auth:null)	ip=192.168.3.159	proxyIp=null	method=GET	uri=/api/v1/sessions/count	protocol=HTTP/1.1	status=401
08:10:47.940 INFO AuthenticationAuditLogger: user=client(auth:NEGOTIATE)	ip=192.168.3.159	proxyIp=null	method=POST	uri=/api/v1/sessions	protocol=HTTP/1.1	status=200
08:10:47.999 INFO AuthenticationAuditLogger: user=client(auth:NEGOTIATE)	ip=192.168.3.159	proxyIp=null	method=DELETE	uri=/api/v1/sessions/86d3e4f5-2739-4759-9320-82a29914ab63	protocol=HTTP/1.1	status=200

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

image

  • Run test locally before make a pull request

@turboFei turboFei force-pushed the batch_log branch 3 times, most recently from 34028bf to 49bec8b Compare December 8, 2022 16:18
@turboFei turboFei changed the title audit rest log Support to audit the authentication http request Dec 8, 2022
@turboFei turboFei self-assigned this Dec 8, 2022
@github-actions github-actions bot added the kind:documentation Documentation is a feature! label Dec 9, 2022
@turboFei turboFei requested review from cfmcgrady and pan3793 December 9, 2022 00:08
@codecov-commenter
Copy link

Codecov Report

Merging #3951 (1f1c313) into master (8eac513) will increase coverage by 0.00%.
The diff coverage is 96.29%.

@@            Coverage Diff            @@
##             master    #3951   +/-   ##
=========================================
  Coverage     51.88%   51.89%           
  Complexity       13       13           
=========================================
  Files           521      522    +1     
  Lines         28834    28857   +23     
  Branches       3862     3862           
=========================================
+ Hits          14961    14974   +13     
- Misses        12500    12509    +9     
- Partials       1373     1374    +1
Impacted Files Coverage Δ
...ver/http/authentication/AuthenticationFilter.scala 91.78% <91.66%> (+1.01%) ⬆️
...ttp/authentication/AuthenticationAuditLogger.scala 100.00% <100.00%> (ø)
...thentication/KyuubiHttpAuthenticationFactory.scala 88.88% <100.00%> (+0.65%) ⬆️
.../kyuubi/server/mysql/constant/MySQLErrorCode.scala 13.84% <0.00%> (-6.16%) ⬇️
...ache/kyuubi/server/mysql/MySQLCommandHandler.scala 77.77% <0.00%> (-4.05%) ⬇️
...ache/kyuubi/server/mysql/MySQLGenericPackets.scala 76.59% <0.00%> (-2.13%) ⬇️
...mon/src/main/scala/org/apache/kyuubi/Logging.scala 43.42% <0.00%> (-1.32%) ⬇️
...he/kyuubi/ha/client/etcd/EtcdDiscoveryClient.scala 68.50% <0.00%> (-0.56%) ⬇️
...org/apache/kyuubi/operation/ExecuteStatement.scala 76.25% <0.00%> (ø)
...apache/kyuubi/service/TBinaryFrontendService.scala 49.46% <0.00%> (+1.07%) ⬆️
... and 1 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@turboFei turboFei closed this in 083fd38 Dec 9, 2022
@turboFei
Copy link
Member Author

turboFei commented Dec 9, 2022

thanks, merged to master

turboFei added a commit that referenced this pull request Feb 21, 2023
@turboFei
[KYUUBI #3951][FOLLOWUP] Audit the rest request params
d288a2b 
### _Why are the changes needed?_

Before:
```
user=anonymous(auth:BASIC) ip=127.0.0.1 proxyIp=null    method=GET      uri=/api/v1/operations/5e286c5d-2880-443f-a4e8-633964dcd699/rowset      protocol=HTTP/1.1       status=200
```

After:

```
user=anonymous(auth:BASIC) ip=127.0.0.1 proxyIp=null    method=GET      uri=/api/v1/operations/5e286c5d-2880-443f-a4e8-633964dcd699/rowset      params=maxrows=2&fetchorientation=FETCH_NEXT       protocol=HTTP/1.1       status=200
```

```
params=maxrows=2&fetchorientation=FETCH_NEXT
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4389 from turboFei/rest_params.

Closes #3951

6ffc1ad [fwang12] comments
61e12b1 [fwang12] nit
0632860 [fwang12] Audit the request params

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
turboFei added a commit that referenced this pull request Feb 21, 2023
@turboFei
[KYUUBI #3951][FOLLOWUP] Audit the rest request params
43ef066 
### _Why are the changes needed?_

Before:
```
user=anonymous(auth:BASIC) ip=127.0.0.1 proxyIp=null    method=GET      uri=/api/v1/operations/5e286c5d-2880-443f-a4e8-633964dcd699/rowset      protocol=HTTP/1.1       status=200
```

After:

```
user=anonymous(auth:BASIC) ip=127.0.0.1 proxyIp=null    method=GET      uri=/api/v1/operations/5e286c5d-2880-443f-a4e8-633964dcd699/rowset      params=maxrows=2&fetchorientation=FETCH_NEXT       protocol=HTTP/1.1       status=200
```

```
params=maxrows=2&fetchorientation=FETCH_NEXT
```
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

- [ ] Add screenshots for manual tests if appropriate

- [x] [Run test](https://kyuubi.readthedocs.io/en/master/develop_tools/testing.html#running-tests) locally before make a pull request

Closes #4389 from turboFei/rest_params.

Closes #3951

6ffc1ad [fwang12] comments
61e12b1 [fwang12] nit
0632860 [fwang12] Audit the request params

Authored-by: fwang12 <fwang12@ebay.com>
Signed-off-by: fwang12 <fwang12@ebay.com>
(cherry picked from commit d288a2b)
Signed-off-by: fwang12 <fwang12@ebay.com>
SusurHe pushed a commit to SusurHe/incubator-kyuubi that referenced this pull request Feb 21, 2023
Merge branch 'master' of github.com:apache/kyuubi
4cd3d23 
* 'master' of github.com:apache/kyuubi: (1557 commits)
  [KYUUBI apache#3951][FOLLOWUP] Audit the rest request params
  [KYUUBI apache#4377] Grant execute permission to release scripts
  [KYUUBI apache#4374] Release uploading should include kyuubi-spark-connector-hive
  [KYUUBI apache#4267] Show warning if SessionHandle is invalid
  [KYUUBI apache#4385] [DOCS] Refine release process
  [KYUUBI apache#4352] Support System.gc() with periodic GC interval
  [KYUUBI apache#4152][FOLLOWUP] LDAP configurations should be server-only
  [KYUUBI apache#4373] Using SVN_STAGING_REPO instead of SVN_STAGING_REPO in the release script to fix echo message
  [KYUUBI apache#4372] Support to return null value for OperationsResource rowset
  [KYUUBI apache#4371] Fix typo in `kyuubi_ecosystem.drawio`
  [KYUUBI apache#4216] Support to transfer client version for kyuubi hive jdbc and rest client sdk
  [KYUUBI apache#4345] Add the doc of kyuubi trino server
  [KYUUBI apache#3081][DOCS] Add Hudi connector doc in Trino
  [KYUUBI apache#4357] Bump Jersey from 2.38 to 2.39
  [KYUUBI apache#4338][FOLLOWUP] Fix K8s integration tests
  [KYUUBI apache#4326] [ARROW] Fix Spark session timezone format in arrow-based result format
  [KYUUBI apache#4360][FOLLOWUP] Get valid unlimited users from existing limiters instead of conf
  [KYUUBI apache#4362] Add `_configurations` in kerberos.rst
  [KYUUBI apache#4338] Bump Spark from 3.3.1 to 3.3.2
  [KYUUBI apache#4119][FOLLOWUP] Add app start time for batch api docs
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pan3793 pan3793 pan3793 approved these changes

@cfmcgrady cfmcgrady Awaiting requested review from cfmcgrady

Assignees

@turboFei turboFei

Labels
kind:documentation Documentation is a feature! module:server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@turboFei @codecov-commenter @pan3793