Enable prototype pollution protection in TSVB #85952

DianaDerevyankina · 2020-12-15T13:06:31Z

Closes #78908

Summary

Rename validateObject function to ensureNoUnsafeProperties
Moved ensureNoUnsafeProperties to @kbn/std package
Used ensureNoUnsafeProperties in TSVB endpoint before the schema validation

Checklist

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

Closes elastic#78908

DianaDerevyankina · 2020-12-15T14:34:38Z

@elasticmachine merge upstream

alexwizp · 2020-12-15T15:09:53Z

src/plugins/vis_type_timeseries/server/routes/vis.ts

@@ -41,6 +42,7 @@ export const visDataRoutes = (
    },
    async (requestContext, request, response) => {
      try {
+        validateObject(request.body);


@flash1293 could you please confirm that it's exactly is what you described in #78908.

Honestly I don't fully understand why we need it cause I've tried to add __proto__, prototype properties into payload and got the following error: "Invalid request payload JSON format"
Just want to be double sure that we really need it

flash1293 · 2020-12-16T14:49:04Z

@elasticmachine merge upstream

flash1293

I tested this and it's not doing what it should. I'm not sure how validateObject should be used (some documentation would be great), but the request should fail if sensitive keys are used in the request body. It seems like something in the stack is already doing this for the __proto__ key, but it didn't seem to catch constructor.

If the validation fails, it shouldn't just be logged but fail the request with a 400 status code.

flash1293 · 2020-12-16T18:21:07Z

I just realized I'm maybe misunderstanding how constructor is handled - @legrego is it possible this basic form of prototype pollution protection is already baked in into the stack for some reason on a lower level and we don't need to catch it in the route handler?

legrego · 2020-12-16T19:08:16Z

I just realized I'm maybe misunderstanding how constructor is handled

constructor on its own isn't problematic, but constructor.prototype is. validateObject should be stopping inputs that contain:

{
  "constructor": { "prototype": "any" }
}

is it possible this basic form of prototype pollution protection is already baked in into the stack for some reason on a lower level and we don't need to catch it in the route handler?

Our newer version of Hapi may be protecting against some cases of prototype pollution, but it doesn't catch everything. It appears to be trying to catch __proto__, but I haven't tested this locally to verify. Based on what I'm seeing, I wouldn't expect Hapi to throw an error when this happens though, so the Invalid request payload JSON format error might be happening for a different reason.

flash1293 · 2020-12-16T19:24:37Z

Thanks @legrego ! @dziyanadzeraviankina , @alexwizp #85952 (review) still holds :)

…bana into Diana/78908

elasticmachine · 2020-12-17T17:29:56Z

Pinging @elastic/kibana-app (Team:KibanaApp)

flash1293

KibanaApp changes LGTM - tested in chrome and __proto__ and constructor.prototype is caught correctly.

@legrego Just to be sure could you take a look as well?

flash1293 · 2020-12-17T17:32:16Z

docs/development/core/server/kibana-plugin-core-server.validateobject.md

+
+[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [validateObject](./kibana-plugin-core-server.validateobject.md)
+
+## validateObject() function


It might make sense to add an explanatory sentence to the documentation of this helper

flash1293 · 2020-12-17T17:38:15Z

I'm not sure whether we have an issue for this, but we discussed to remove the soft validation which is still in there. I guess we will tackle that on a separate PR?

pgayvallet · 2020-12-17T19:20:52Z

src/core/server/http/index.ts

@@ -94,3 +94,4 @@ export {
 } from './cookie_session_storage';
 export * from './types';
 export { BasePath, IBasePath } from './base_path_service';
+export { validateObject } from './prototype_pollution';


I would really like to avoid static functions leaking outside of core. If this needs to be used in plugins, maybe moving it to a package (existing or new) would make sense. wdyt @legrego?

I'm not opposed to moving this into a package. That'd certainly make it easier to consume

@dziyanadzeraviankina we are currently discussing the details with @legrego. Note that if it's mandatory to have this merged for 7.11, we could do the moving/refactoring at a later time. So considerer this approved on my side if this is urgent.

I'm not sure if there's a suitable package for that, maybe it could be moved to kbn-config-schema package, or is it better to create a new one?

Would @kbn/std work for this?

legrego

Naming aside, LGTM!

Exporting a function named validateObject from a package named @kbn/std is really vague. It's not clear from reading this what the purpose of validateObject is. That said, naming is hard so don't consider this a blocker. One idea: ensureNoUnsafeProperties

alexwizp

LGTM

sulemanof

LGTM 👍

DianaDerevyankina · 2020-12-21T10:43:15Z

@pgayvallet could you review the changes, please?

flash1293 · 2020-12-21T16:24:50Z

Still LGTM from the KibanaApp side :)

mshustov

ok for the core team changes

alexwizp · 2020-12-30T11:36:55Z

@elasticmachine merge upstream

kibanamachine · 2020-12-30T13:46:42Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 59a0e40

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`core`	448	449	+1
`data`	611	612	+1
`fleet`	457	458	+1
`graph`	170	171	+1
`urlDrilldown`	28	29	+1
total			+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`data`	268.1KB	270.8KB	+2.7KB
`fleet`	964.6KB	967.2KB	+2.7KB
`graph`	1.3MB	1.3MB	+2.7KB
total			+8.1KB

Distributable file count

id	before	after	diff
`default`	48027	48026	-1
`oss`	27714	27713	-1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`core`	556.2KB	558.9KB	+2.7KB
`urlDrilldown`	51.2KB	53.9KB	+2.7KB
total			+5.4KB

History

💚 Build #95675 succeeded 8f1f8a9
💚 Build #95442 succeeded 7b9aff0
💔 Build #95414 failed 0555f02
💚 Build #95109 succeeded 679417c
💚 Build #94889 succeeded 01711fd

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* Enable prototype pollution protection in TSVB Closes elastic#78908 * Update Dock API Changes * Replace logging failed in validateObject validation with 400 error * Move validateObject to kbn-std package and add a description * Update Doc API Changes * Rename validateObject function to ensureNoUnsafeProperties * Rename other validateObject occurrences Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* Enable prototype pollution protection in TSVB Closes #78908 * Update Dock API Changes * Replace logging failed in validateObject validation with 400 error * Move validateObject to kbn-std package and add a description * Update Doc API Changes * Rename validateObject function to ensureNoUnsafeProperties * Rename other validateObject occurrences Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Diana Derevyankina <54894989+DziyanaDzeraviankina@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

…meline-component * 'master' of github.com:elastic/kibana: (955 commits) remove SameSite:None workaround (elastic#86994) URL encoding for URL drilldown (elastic#86902) [Security Solution] Fix few styling issues (elastic#87045) [APM] Custom links can still be created with a read only user. (elastic#87089) prevent double update (elastic#86794) Upgrade @hapi/hoek to revert hack introduced in hapi v20 upgrade (elastic#87113) [APM] Every time the new Header Icon is clicked it fetches data (elastic#87093) [APM] Add range query to service map trace walk (elastic#86631) [Discover] Deangularize navbar in context app (elastic#86353) skip "should schedule actions on legacy alerts" elastic#87010 🍾 update notice text for 2021 [logstash] remove "upgrade" functionality now that .logstash is a system index (elastic#87056) Enable prototype pollution protection in TSVB (elastic#85952) [Security Solution] add a consistent spelling of ES in Policy Response (elastic#87073) [SECURITY_SOLUTION][ENDPOINT] Delete Endpoint Policy List code (elastic#87063) Adds more URLs to the docs links service (elastic#86972) Add missing backticks in reporting-settings.asciidoc (elastic#77979) [test/functional_cors] 9000 is sometimes in use, make getPort random (elastic#87050) [Security Solution] Fix Timeline filter EuiSuperSelect styling (elastic#87033) [Lens] Fix duplicate suggestions on single-bucket charts (elastic#86996) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/index.ts

Enable prototype pollution protection in TSVB

2724105

Closes elastic#78908

DianaDerevyankina added Feature:TSVB TSVB (Time Series Visual Builder) Team:Visualizations Visualization editors, elastic-charts and infrastructure v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.11.0 labels Dec 15, 2020

DianaDerevyankina requested review from VladLasitsa, alexwizp and sulemanof December 15, 2020 13:06

DianaDerevyankina self-assigned this Dec 15, 2020

kibanamachine and others added 2 commits December 15, 2020 09:34

Merge branch 'master' into Diana/78908

8f12f42

Merge branch 'master' into Diana/78908

389eb9c

alexwizp reviewed Dec 15, 2020

View reviewed changes

alexwizp requested a review from flash1293 December 15, 2020 15:10

DianaDerevyankina added 2 commits December 16, 2020 10:10

Merge branch 'master' into Diana/78908

9b74da8

Update Dock API Changes

631bd57

Merge branch 'master' into Diana/78908

01711fd

flash1293 requested changes Dec 16, 2020

View reviewed changes

DianaDerevyankina added 3 commits December 17, 2020 15:03

Merge branch 'master' into Diana/78908

0e2ce95

Merge branch 'Diana/78908' of https://github.com/DianaDerevyankina/ki…

e773319

…bana into Diana/78908

Replace logging failed in validateObject validation with 400 error

679417c

flash1293 marked this pull request as ready for review December 17, 2020 17:29

flash1293 requested a review from a team December 17, 2020 17:29

flash1293 requested a review from a team as a code owner December 17, 2020 17:29

flash1293 approved these changes Dec 17, 2020

View reviewed changes

flash1293 reviewed Dec 17, 2020

View reviewed changes

pgayvallet reviewed Dec 17, 2020

View reviewed changes

DianaDerevyankina added 4 commits December 18, 2020 10:26

Merge branch 'master' into Diana/78908

82cff88

Move validateObject to kbn-std package and add a description

0555f02

Merge branch 'master' into Diana/78908

4160164

Update Doc API Changes

7b9aff0

legrego approved these changes Dec 18, 2020

View reviewed changes

DianaDerevyankina added 3 commits December 21, 2020 10:33

Merge branch 'master' into Diana/78908

b9185ee

Rename validateObject function to ensureNoUnsafeProperties

4261fbe

Rename other validateObject occurrences

8f1f8a9

alexwizp approved these changes Dec 21, 2020

View reviewed changes

sulemanof approved these changes Dec 21, 2020

View reviewed changes

DianaDerevyankina requested a review from pgayvallet December 22, 2020 07:55

mshustov approved these changes Dec 30, 2020

View reviewed changes

Merge branch 'master' into Diana/78908

59a0e40

alexwizp merged commit aecee92 into elastic:master Dec 31, 2020

alexwizp mentioned this pull request Dec 31, 2020

[7.x] Enable prototype pollution protection in TSVB (#85952) #87087

Merged

alexwizp added v7.12.0 and removed v7.11.0 labels Dec 31, 2020

wylieconlon mentioned this pull request May 5, 2021

@kbn/config-schema is slow #78351

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable prototype pollution protection in TSVB #85952

Enable prototype pollution protection in TSVB #85952

DianaDerevyankina commented Dec 15, 2020 •

edited

Loading

DianaDerevyankina commented Dec 15, 2020

alexwizp Dec 15, 2020

flash1293 commented Dec 16, 2020

flash1293 left a comment

flash1293 commented Dec 16, 2020

legrego commented Dec 16, 2020

flash1293 commented Dec 16, 2020

elasticmachine commented Dec 17, 2020

flash1293 left a comment

flash1293 Dec 17, 2020

flash1293 commented Dec 17, 2020

pgayvallet Dec 17, 2020

legrego Dec 18, 2020

pgayvallet Dec 18, 2020

DianaDerevyankina Dec 18, 2020

legrego Dec 18, 2020

legrego left a comment

alexwizp left a comment

sulemanof left a comment

DianaDerevyankina commented Dec 21, 2020

flash1293 commented Dec 21, 2020

mshustov left a comment

alexwizp commented Dec 30, 2020

kibanamachine commented Dec 30, 2020


		[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [validateObject](./kibana-plugin-core-server.validateobject.md)

		## validateObject() function

Enable prototype pollution protection in TSVB #85952

Enable prototype pollution protection in TSVB #85952

Conversation

DianaDerevyankina commented Dec 15, 2020 • edited Loading

Summary

Checklist

For maintainers

DianaDerevyankina commented Dec 15, 2020

alexwizp Dec 15, 2020

Choose a reason for hiding this comment

flash1293 commented Dec 16, 2020

flash1293 left a comment

Choose a reason for hiding this comment

flash1293 commented Dec 16, 2020

legrego commented Dec 16, 2020

flash1293 commented Dec 16, 2020

elasticmachine commented Dec 17, 2020

flash1293 left a comment

Choose a reason for hiding this comment

flash1293 Dec 17, 2020

Choose a reason for hiding this comment

flash1293 commented Dec 17, 2020

pgayvallet Dec 17, 2020

Choose a reason for hiding this comment

legrego Dec 18, 2020

Choose a reason for hiding this comment

pgayvallet Dec 18, 2020

Choose a reason for hiding this comment

DianaDerevyankina Dec 18, 2020

Choose a reason for hiding this comment

legrego Dec 18, 2020

Choose a reason for hiding this comment

legrego left a comment

Choose a reason for hiding this comment

alexwizp left a comment

Choose a reason for hiding this comment

sulemanof left a comment

Choose a reason for hiding this comment

DianaDerevyankina commented Dec 21, 2020

flash1293 commented Dec 21, 2020

mshustov left a comment

Choose a reason for hiding this comment

alexwizp commented Dec 30, 2020

kibanamachine commented Dec 30, 2020

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Distributable file count

Page load bundle

History

DianaDerevyankina commented Dec 15, 2020 •

edited

Loading