Integrate new platform (core) server side into Kibana #18951

azasypkin · 2018-05-09T14:09:27Z

In this PR we're integrating base parts of the new platform (later core) server side into Kibana. At this point there are only two major touch points between new core and the rest of Kibana:

Now it's the core who listens on HTTP ports, handles TLS configuration and base path proxy. All requests to Kibana will hit HTTP server exposed by the core first and it will decide whether request can be solely handled by the new platform or request should be forwarded to the legacy Kibana. Technically we have two Hapi servers now (one for core and one for legacy Kibana), but only one underlying Node HTTP listener. This setup allows us to gradually introduce any "pre-route" processing code, expose new routes or replace old ones handled by the legacy platform currently.
Once config has been validated by the legacy Kibana, part of it will also be validated by the new core so that we can make config validation stricter with the new config validation system (even though it's based on Joi internally we introduced custom rules tailored to our needs like byteSize or duration). That means you can notice new config validation error messages if config value has been accepted by legacy config validation code, but rejected by the core.

With this PR legacy Kibana will still be started first using existing CLI and will bootstrap core only when needed, but we're going to inverse this in #19994.

New platform formerly resided in new-platform feature branch and then major part of it has been migrated to kbn-core feature branch. new-platform feature branch is going to stay for a while so that we can migrate the missing parts when we're ready. Here is a list of notable kbn-core vs new-platform differences:

Removed plugins supporting code, Elasticsearch client, cli
Reorganized platform (now core) directory structure according to Directory structure with the new platform #12466
Made kbn-observable package a part of the core and moved it to src/core/lib to use relative paths, removed unused operators
Transformed @kbn/utils into src/core/server/config/schema and based it on Joi internally

…hem. (#12662)

…lasticsearch and crypto modules. (#12680)

#12687) * Rename 'xpack_api_polling_frequency_millis' to 'api_polling_frequency' * prettier

* Cleaner Pid observable code * remove unused arg

* Implement mapOf setting type * better variable names * sharing code

epixa

This was reviewed over the last year in a long series of individual PRs. I just did a high level once over on the code to double check, and it looks good.

Please update the PR title and description though. To most people, this PR represents the first addition of the new platform. Can you update this PR to detail the changes?

LGTM

rhoboat · 2018-07-10T13:10:16Z

Question about the kbn-observable that we changed into src/core/lib/kbn_observable: why didn't we move the share_last operator over? Should we make a note about this, in case we plan to reimplement that? I assume we don't want it anymore at all?

rhoboat

Can we either add some follow up about the stuff that we didn't pull over from kbn-observable, or include them here? If this gets merged as is, the things that didn't make it will be lost/forgotten.

epixa · 2018-07-10T18:07:20Z

@archanid Are you referring to kbn-observable functionality from the new-platform branch that wasn't included here? If so, that shouldn't block this change.

The new-platform branch was abandoned the moment we created kbn-core, it continues to exist only so we can reference and potentially scavenge pieces from it in the future if we need to. So if there is a feature from it that you'd like to use going forward, you can copy it from that branch and PR to master just like any other change.

rhoboat · 2018-07-10T18:15:20Z

@epixa yes, indeed.
Hm, but why didn't we include everything but that one operator? I just wonder. The description says we pulled the whole kbn-observable lib in here. Is it missing, or deliberately omitted?

epixa · 2018-07-10T18:17:19Z

@archanid Our stance with code has always been to delete it if it's not being used, and it's not being used. I'm not sure if that's why it was omitted, but it's why I would have omitted it had I made the change :)

azasypkin · 2018-07-10T18:45:50Z

Question about the kbn-observable that we changed into src/core/lib/kbn_observable: why didn't we move the share_last operator over? Should we make a note about this, in case we plan to reimplement that? I assume we don't want it anymore at all?

I just removed everything we don't need in the initial merge, and it was the only operator that depended on RxJS, but not used elsewhere. Since we have this in new-platform (as well as other pieces like elasticsearch client, plugins support etc.) we can pull it in when/if we need it.

epixa · 2018-07-10T19:25:06Z

@archanid woo!

@azasypkin Please update the title/description and fix the dependency conflicts and get this beautiful beast merged!

elasticmachine · 2018-07-11T08:12:26Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

Co-authored-by: Kim Joar Bekkelund <kjbekkelund@gmail.com> Co-authored-by: archana <archanid@users.noreply.github.com> Co-authored-by: Spencer <spalger@users.noreply.github.com> Co-authored-by: Court Ewing <court@epixa.com>

azasypkin · 2018-07-11T10:12:51Z

6.x/6.4: f88d0b9

elasticmachine · 2018-07-11T10:46:24Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

azasypkin · 2018-07-11T10:54:22Z

Hmm, why CI is still running for this PR? :) Anyway it seems CI failed because of flaky test... Will be monitoring master today.

w33ble · 2018-07-13T22:40:45Z

This PR broke Canvas. We just see a bunch of 404 errors in the console and nothing loads. There's an extra slash in the basepath now, not sure if that's related.

I opened #20802

kimjoar and others added 30 commits June 20, 2017 23:26

New platform initial setup (#11468)

1cb9397

Merge branch 'master' of github.com:elastic/kibana into new-platform

10516ec

Merge branch 'master' into new-platform

4048c6a

Make config types easier to read

cc374da

Add a branded types helper, use for PluginName

c62bda2

SavedObjectsFacade -> SavedObjectsService

51ca2a1

Improve comments and code for topological sort

03130b7

Improve comments related to plugin system

e3e8593

Make starting and stopping core services async

f93e80f

Rename from 'kibana' to 'core' for clarity

c5d1770

Transform argv to config overrides

b7235ca

Move process.exit to cli

e9c43aa

Define plugin as object instead of separate params

8a503fd

Move configPath to top-level field on plugins

1d67e06

Only attempt to stop plugins that have been started

455bd81

Handle 'flat' config yaml

3f1dd1d

Don't start disabled plugins

d2a4257

Extract RawConfigService and inject config observable into ConfigService

93adf4b

Change to specifying log level instead of quiet, silent, verbose

bc6cd7a

Load config _before_ starting 'Root'

89af7ba

Correctly handle array config values and expand flat objects within t…

7f0cfac

…hem. (#12662)

Add (and run) Prettier on the new platform code (#12667)

2c50926

Upgrade Typescript, RxJS and type definitions. Add augmentation for e…

c0abc41

…lasticsearch and crypto modules. (#12680)

Extract PidFile and clarify rxjs use in PidService

a2b15c3

Comment out and clean transitive dep example

465130b

Typo in PidService

a7a1afb

Better explanation of 'switchMap'

627674e

Rename 'xpack_api_polling_frequency_millis' to 'api_polling_frequency' (

243dcd7

#12687) * Rename 'xpack_api_polling_frequency_millis' to 'api_polling_frequency' * prettier

Cleaner Pid observable code (#12702)

d287ca9

* Cleaner Pid observable code * remove unused arg

Implement mapOf setting type (#12703)

57951f6

* Implement mapOf setting type * better variable names * sharing code

azasypkin requested a review from spalger June 29, 2018 22:01

epixa approved these changes Jul 9, 2018

View reviewed changes

rhoboat suggested changes Jul 10, 2018

View reviewed changes

rhoboat approved these changes Jul 10, 2018

View reviewed changes

azasypkin added 2 commits July 11, 2018 08:47

Merge branch 'master' into kbn-core

e341c55

Prettier: apply latest printWidth rule.

49d6338

azasypkin changed the title ~~Merge kbn-core into master~~ Integrate new platform (core) server side into Kibana Jul 11, 2018

azasypkin merged commit 1418310 into master Jul 11, 2018

azasypkin mentioned this pull request Jul 11, 2018

[6.x] Integrate new platform (core) server side into Kibana #20661

Merged

azasypkin added backport pending v6.4.0 labels Jul 11, 2018

azasypkin added backported and removed backport pending labels Jul 11, 2018

azasypkin mentioned this pull request Jul 16, 2018

New Platform broke Canvas #20802

Closed

azasypkin deleted the kbn-core branch July 24, 2018 15:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate new platform (core) server side into Kibana #18951

Integrate new platform (core) server side into Kibana #18951

azasypkin commented May 9, 2018 •

edited

Loading

epixa left a comment

rhoboat commented Jul 10, 2018

rhoboat left a comment

epixa commented Jul 10, 2018

rhoboat commented Jul 10, 2018

epixa commented Jul 10, 2018

azasypkin commented Jul 10, 2018 •

edited

Loading

epixa commented Jul 10, 2018

elasticmachine commented Jul 11, 2018

azasypkin commented Jul 11, 2018

elasticmachine commented Jul 11, 2018

azasypkin commented Jul 11, 2018 •

edited

Loading

w33ble commented Jul 13, 2018 •

edited

Loading

Integrate new platform (core) server side into Kibana #18951

Integrate new platform (core) server side into Kibana #18951

Conversation

azasypkin commented May 9, 2018 • edited Loading

epixa left a comment

Choose a reason for hiding this comment

rhoboat commented Jul 10, 2018

rhoboat left a comment

Choose a reason for hiding this comment

epixa commented Jul 10, 2018

rhoboat commented Jul 10, 2018

epixa commented Jul 10, 2018

azasypkin commented Jul 10, 2018 • edited Loading

epixa commented Jul 10, 2018

elasticmachine commented Jul 11, 2018

💚 Build Succeeded

azasypkin commented Jul 11, 2018

elasticmachine commented Jul 11, 2018

💔 Build Failed

azasypkin commented Jul 11, 2018 • edited Loading

w33ble commented Jul 13, 2018 • edited Loading

azasypkin commented May 9, 2018 •

edited

Loading

azasypkin commented Jul 10, 2018 •

edited

Loading

azasypkin commented Jul 11, 2018 •

edited

Loading

w33ble commented Jul 13, 2018 •

edited

Loading