Background search service POC #64641

lizozom · 2020-04-28T12:16:15Z

Summary

Summarize your PR. If it involves visual changes include a screenshot or gif.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser
This was checked for cross-browser compatibility, including a check against IE11

For maintainers

This was checked for breaking API changes and was labeled appropriately

lizozom · 2020-04-28T12:18:02Z

src/plugins/data/server/search/i_search_context.ts

@@ -23,4 +23,6 @@ import { CoreSetup, SharedGlobalConfig } from '../../../../core/server';
 export interface ISearchContext {
  core: CoreSetup;
  config$: Observable<SharedGlobalConfig>;
+  backgroundSearchService?: any;


How to extend context??

@lukasolson

lizozom · 2020-04-28T12:18:11Z

src/plugins/data/public/search/search_source/search_source.ts

@@ -139,6 +140,15 @@ export class SearchSource {
    return { ...this.fields };
  }

+  getSessionId(): string | undefined {


Can be private

lizozom · 2020-04-28T12:18:33Z

src/plugins/data/server/search/routes.ts

+        const response = await context.search!.search(
+          searchRequest,
+          { signal, rawRequest: request },
+          strategy


Should strategy be a part of options?

FWIW it would make sense as an option to me instead of its own argument (and uses the default strategy if left empty), but 🤷‍♂️

lizozom · 2020-04-28T12:19:19Z

x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts

+    const user = context?.security?.authc.getCurrentUser(options.rawRequest);
+    if (user) {
+      context.backgroundSearchService.trackId(
+        user.email,


Is user.email the best info to use (in terms of uniqueness)?

lizozom · 2020-04-28T12:19:26Z

x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts

@@ -71,14 +106,16 @@ async function asyncSearch(
  const path = encodeURI(request.id ? `/_async_search/${request.id}` : `/${index}/_async_search`);

  // Wait up to 1s for the response to return
-  const query = toSnakeCase({ waitForCompletionTimeout: '1s', ...queryParams });
+  const query = toSnakeCase({ waitForCompletionTimeout: '1ms', ...queryParams });


src/plugins/data/public/search/expressions/esaggs.ts

lizozom · 2020-04-30T07:24:51Z

@elasticmachine merge upstream

…ound-search-service

Unit tests for background_session_service Remove (temporarily) version handling

…ound-search-service

…ng notifucation, standalone session service

…ound-search-service

Add session ID to dashboard URL Reduce store interval

Added better notifications Added basic docs to service

…ound-search-service

x-pack/plugins/data_enhanced/public/background_session/background_session_service.ts

…ound-search-service

…vice

…ound-search-service

lizozom · 2020-08-11T20:53:57Z

src/plugins/data/public/search/search_interceptor.ts

@@ -98,7 +83,7 @@ export class SearchInterceptor {
  ): Observable<IEsSearchResponse> {
    const { id, ...searchRequest } = request;
    const path = trimEnd(`/internal/search/${strategy || ES_SEARCH_STRATEGY}/${id || ''}`, '/');
-    const body = JSON.stringify(id != null ? {} : searchRequest);
+    const body = JSON.stringify(id != null ? { stored: request.stored } : searchRequest);


TODO: Need to make sure that searchRequest has the correct shape, rather than making these decidions in the OSS interceptor

…ound-search-service

…-search-service

…ound-search-service

kibanamachine · 2020-08-17T09:23:18Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: c46e79e
Pipeline Steps (look for red circles / failed steps)
Interpreting CI Failures

Failed CI Steps

Test Failures

X-Pack Jest Tests.x-pack/plugins/data_enhanced/public/search.EnhancedSearchInterceptor search should resolve immediately if first call returns full result

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: this.deps.session.getStored is not a function
    at EnhancedSearchInterceptor.search (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.ts:80:40)
    at Object.test (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.test.ts:103:42)
    at Promise (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:198:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:162:10)
    at _callCircusTest (/dev/shm/workspace/kibana/node_modules/jest-circus/build/run.js:205:40)
    at process._tickCallback (internal/process/next_tick.js:68:7)

X-Pack Jest Tests.x-pack/plugins/data_enhanced/public/search.EnhancedSearchInterceptor search should make secondary request if first call returns partial result

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: this.deps.session.getStored is not a function
    at EnhancedSearchInterceptor.search (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.ts:80:40)
    at Object.test (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.test.ts:141:42)
    at Promise (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:198:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:162:10)
    at _callCircusTest (/dev/shm/workspace/kibana/node_modules/jest-circus/build/run.js:205:40)
    at process._tickCallback (internal/process/next_tick.js:68:7)

X-Pack Jest Tests.x-pack/plugins/data_enhanced/public/search.EnhancedSearchInterceptor search should abort if request is partial and not running (ES graceful error)

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

TypeError: this.deps.session.getStored is not a function
    at EnhancedSearchInterceptor.search (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.ts:80:40)
    at Object.test (/dev/shm/workspace/parallel/10/kibana/x-pack/plugins/data_enhanced/public/search/search_interceptor.test.ts:172:42)
    at Promise (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:198:28)
    at new Promise (<anonymous>)
    at callAsyncCircusFn (/dev/shm/workspace/kibana/node_modules/jest-circus/build/utils.js:162:10)
    at _callCircusTest (/dev/shm/workspace/kibana/node_modules/jest-circus/build/run.js:205:40)
    at process._tickCallback (internal/process/next_tick.js:68:7)

and 12 more failures, only showing the first 3.

Build metrics

@kbn/optimizer bundle module count

id	value	diff	baseline
data	545	+5	540
dataEnhanced	175	+157	18
total		+162

page load bundle size

id	value	diff	baseline
dashboard	689.7KB	+2.0KB	687.7KB
data	1.4MB	+4.6KB	1.4MB
dataEnhanced	604.9KB	⚠️ +426.9KB	178.0KB
total		+433.5KB

Saved Objects .kibana field count

id	value	diff	baseline
background-session	8	+8	-

History

💔 Build #67906 failed de22da1
💔 Build #67901 failed 6687554
💔 Build #67683 failed 8851c52
💔 Build #67606 failed 814f63c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Liza K added 4 commits April 27, 2020 20:41

uuid function

5cf724d

Track async search IDs on server

d0680a1

stabilize TS

e231ea2

Use bg search from dashboard and esaggs

88a65dc

lizozom self-assigned this Apr 28, 2020

lizozom commented Apr 28, 2020

View reviewed changes

lizozom requested a review from lukasolson April 28, 2020 12:19

Commit bg search service

99b4e96

lizozom commented Apr 28, 2020

View reviewed changes

src/plugins/data/public/search/expressions/esaggs.ts Outdated Show resolved Hide resolved

lizozom added the v8.0.0 label Apr 30, 2020

Liza K added 14 commits April 30, 2020 20:30

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

0487899

…ound-search-service

fix ts

118572b

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

8154930

…ound-search-service

Background Session saved object

4b47a3f

Use sessionID to restore session

cf9d368

Unit tests for background_session_service Remove (temporarily) version handling

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

f939ccd

…ound-search-service

Improve service stability, add smarter handling of showing long runni…

6ca9b02

…ng notifucation, standalone session service

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

5cc76b4

…ound-search-service

Notify on user sending to background using observable

3af9316

Add session ID to dashboard URL Reduce store interval

Fixed strategy logic

43058e3

Added better notifications Added basic docs to service

Background session example plugin

eec29d2

Restore hiding toast

c351106

improve example code

47328b1

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

2a7447e

…ound-search-service

lizozom commented May 21, 2020

View reviewed changes

x-pack/plugins/data_enhanced/public/background_session/background_session_service.ts Outdated Show resolved Hide resolved

Liza K added 20 commits July 28, 2020 16:54

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

3aa5f31

…ound-search-service

jest and i18n

516cdeb

Merge branch 'master' of github.com:elastic/kibana

e5b2c84

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

65e6e74

…ound-search-service

Pass in session to object update

004b89a

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

f72596d

…ound-search-service

handle expiration updates

15a09f3

Merge branch 'master' of github.com:elastic/kibana

7e767cb

Merge branch 'master' of github.com:elastic/kibana

5a3dfea

Enhance pattern

10847b1

fix tests

2885c2d

Merge branch 'data/enhance-pattern' into search/background-search-ser…

9925725

…vice

fix test

d8383db

Merge branch 'data/enhance-pattern' into search/background-search-ser…

cd5ae44

…vice

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

a5075b0

…ound-search-service

Move notification to top nav and cleanup code

f33dce2

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

512eb99

…ound-search-service

Pass search ids from FE

b44e3c3

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

233e1c8

…ound-search-service

Delete background session example

814f63c

lizozom commented Aug 11, 2020

View reviewed changes

Liza K added 7 commits August 12, 2020 12:03

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

f9edabe

…ound-search-service

Type fixes

fb1669c

Simplify background session service

3a35568

docs and translations

8851c52

Merge remote-tracking branch 'upstream/master' into search/background…

6687554

…-search-service

typos

de22da1

Merge branch 'master' of github.com:elastic/kibana into search/backgr…

c46e79e

…ound-search-service

lizozom closed this Nov 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Background search service POC #64641

Background search service POC #64641

lizozom commented Apr 28, 2020

lizozom Apr 28, 2020

lizozom Apr 28, 2020

lizozom Apr 28, 2020

lizozom Apr 28, 2020

lukeelmers Jun 1, 2020

lizozom Apr 28, 2020

lizozom Apr 28, 2020

lizozom commented Apr 30, 2020

lizozom Aug 11, 2020

kibanamachine commented Aug 17, 2020 •

edited

Loading

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Background search service POC #64641

Background search service POC #64641

Conversation

lizozom commented Apr 28, 2020

Summary

Checklist

For maintainers

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lizozom commented Apr 30, 2020

Choose a reason for hiding this comment

kibanamachine commented Aug 17, 2020 • edited Loading

💔 Build Failed

Failed CI Steps

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Build metrics

@kbn/optimizer bundle module count

page load bundle size

Saved Objects .kibana field count

History

kibanamachine commented Aug 17, 2020 •

edited

Loading