Skip to content

Commit

Permalink
[Console] Add logic for query params (elastic#160515)
Browse files Browse the repository at this point in the history 
## Summary

Fixes elastic#160528 
Follow up to elastic#159241 

This PR adds logic for query parameters to the new script generating
Console autocomplete definitions from ES specification. The logic is
explained in details in the
[file](https://github.com/elastic/kibana/pull/160515/files#diff-b6853462c38db4e237dbb3cdec9d9f6659aa3fdc5f96a6193f2c4bac1439db43)
but here is a short version:
- Currently, the autocomplete engine only works with url params of 2
types: boolean (`__flag__`) and a list of options (for example `['all',
'open', 'hidden']`). The script will convert all of the url params from
the specification into this format: a boolean or a list. If there are no
set options for a parameter, but a default value is known, then it will
be converted into a list with 1 option, for example `['random']` so that
the autocomplete engine will display it as a single suggestion. We also
need to convert any numbers to strings, because they won't be displayed
otherwise.
- Endpoints in the specification have a property `request` which in turn
has 2 properties describing url params: `attachedBehaviours` and
`query`. Both object contain an array of `property`'s each describing a
url param. `property` is configured with either a built in type
(`string`, `number`, `boolean`) or defined type, for example
`ExpandWildcards`. By finding the type `ExpandWildcards` in the
specification, we can convert this type to a list of options `['open',
'all', 'none', 'hidden']`.

### How to test
Similar to elastic#159241, you need to
re-generenate the definitions and see if local changes to definitions
files make sense.
1. Checkout the ES specification
[repo](https://github.com/elastic/elasticsearch-specification)
2. Run the command `node scripts/generate_console_definitions.js
--source <ES_SPECIFICATION_REPO> --emptyDest`
3. Check the changes in the folder
`KIBANA_REPO/src/plugins/console/server/lib/spec_definitions/json/generated`

#### Intended changes to the definitions files
- Most of endpoints have 4 default url params that previously were not
in the definitions files but added to all endpoints in this
[file](https://github.com/elastic/kibana/blob/main/src/plugins/console/public/lib/autocomplete/url_params.js).
These params are configured in the interface `CommonQueryParameters` in
the specification (see this
[file](https://github.com/elastic/elasticsearch-specification/blob/main/specification/_spec_utils/behaviors.ts)).
<details>
The interface in the specification

```js
export interface CommonQueryParameters {
  error_trace?: boolean
  filter_path?: string | string[]
  human?: boolean
  pretty?: boolean
}
```
The converted url params

```json
"error_trace": "__flag__",
"filter_path": [],
"human": "__flag__",
"pretty": "__flag__",
```
</details>

- Previously existing `url_components` property in the definitions is
deleted and this change will be addressed separately. (not sure it is
currently working but added a task to the meta issue)
- Previously numbers were configured as `0` or `0.0` but that is not
currently displayed in suggestions. Instead, the new script converts
numbers to strings and if any default value is present, it will be
displayed as a suggestion.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
  • Loading branch information
@yuliacech @kibanamachine
yuliacech and kibanamachine authored Jul 5, 2023
1 parent 54dc40f commit ee6ca65
Show file tree
Hide file tree
Showing 8 changed files with 1,131 additions and 69 deletions.
11 changes: 9 additions & 2 deletions packages/kbn-generate-console-definitions/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
# @kbn/generate-console-definitions
# Generate console definitions
This package is a script to generate definitions used in Console to display autocomplete suggestions. The script is
a new implementation of `kbn-spec-to-console` package: The old script uses [JSON specs](https://github.com/elastic/elasticsearch/tree/main/rest-api-spec) from the Elasticsearch repo as the source, whereas this script uses the Elasticsearch specification [repo](https://github.com/elastic/elasticsearch-specification) as the source.

## Instructions
1. Checkout the Elasticsearch specification [repo](https://github.com/elastic/elasticsearch-specification).
2. Run the command `node scripts/generate_console_definitions.js --source <ES_SPECIFICATION_REPO> --emptyDest`
This command will use the folder `<ES_SPECIFICATION_REPO>` as the source and the constant [`AUTOCOMPLETE_DEFINITIONS_FOLDER`](https://github.com/elastic/kibana/blob/main/src/plugins/console/common/constants/autocomplete_definitions.ts) as the destination. Based on the value of the constant, the autocomplete definitions will be generated in the folder `<KIBANA_REPO>/src/plugins/server/lib/spec_definitions/json/generated`. Using the flag `--emptyDest` will remove any existing files in the destination folder.
3. It's possible to generate the definitions into a different folder. For that pass an option to the command `--dest <DEFINITIONS_FOLDER>` and also update the constant [`AUTOCOMPLETE_DEFINITIONS_FOLDER`](https://github.com/elastic/kibana/blob/main/src/plugins/console/common/constants/autocomplete_definitions.ts) so that the Console server will load the definitions from this folder.

Empty package generated by @kbn/generate
98 changes: 31 additions & 67 deletions packages/kbn-generate-console-definitions/src/generate_console_definitions.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,51 +9,16 @@
import fs from 'fs';
import Path, { join } from 'path';
import { ToolingLog } from '@kbn/tooling-log';

interface EndpointRequest {
name: string;
namespace: string;
}

interface Endpoint {
name: string;
urls: Array<{
methods: string[];
path: string;
}>;
docUrl: string;
request: null | EndpointRequest;
}

interface SchemaType {
name: {
name: string;
namespace: string;
};
}

interface Schema {
endpoints: Endpoint[];
types: SchemaType[];
}

interface UrlParams {
[key: string]: number | string;
}

interface BodyParams {
[key: string]: number | string;
}

interface Definition {
documentation?: string;
methods: string[];
patterns: string[];
url_params?: UrlParams;
data_autocomplete_rules?: BodyParams;
}

const generateMethods = (endpoint: Endpoint): string[] => {
import { generateQueryParams } from './generate_query_params';
import type {
AutocompleteBodyParams,
AutocompleteDefinition,
AutocompleteUrlParams,
SpecificationTypes,
} from './types';
import { findTypeDefinition } from './utils';

const generateMethods = (endpoint: SpecificationTypes.Endpoint): string[] => {
// this array consists of arrays of strings
const methodsArray = endpoint.urls.map((url) => url.methods);
// flatten to return array of strings
Expand All @@ -62,7 +27,7 @@ const generateMethods = (endpoint: Endpoint): string[] => {
return [...new Set(flattenMethodsArray)];
};

const generatePatterns = (endpoint: Endpoint): string[] => {
const generatePatterns = (endpoint: SpecificationTypes.Endpoint): string[] => {
return endpoint.urls.map(({ path }) => {
let pattern = path;
// remove leading / if present
Expand All @@ -73,42 +38,37 @@ const generatePatterns = (endpoint: Endpoint): string[] => {
});
};

const generateDocumentation = (endpoint: Endpoint): string => {
const generateDocumentation = (endpoint: SpecificationTypes.Endpoint): string => {
return endpoint.docUrl;
};

const generateParams = (
endpoint: Endpoint,
schema: Schema
): { urlParams: UrlParams; bodyParams: BodyParams } | undefined => {
endpoint: SpecificationTypes.Endpoint,
schema: SpecificationTypes.Model
): { urlParams: AutocompleteUrlParams; bodyParams: AutocompleteBodyParams } | undefined => {
const { request } = endpoint;
if (!request) {
return;
}
const requestType = schema.types.find(
({ name: { name, namespace } }) => name === request.name && namespace === request.namespace
);
const requestType = findTypeDefinition(schema, request);
if (!requestType) {
return;
}

const urlParams = generateUrlParams(requestType);
const urlParams = generateQueryParams(requestType as SpecificationTypes.Request, schema);
const bodyParams = generateBodyParams(requestType);
return { urlParams, bodyParams };
};

const generateUrlParams = (requestType: SchemaType): UrlParams => {
return {};
};

const generateBodyParams = (requestType: SchemaType): BodyParams => {
const generateBodyParams = (
requestType: SpecificationTypes.TypeDefinition
): AutocompleteBodyParams => {
return {};
};

const addParams = (
definition: Definition,
params: { urlParams: UrlParams; bodyParams: BodyParams }
): Definition => {
definition: AutocompleteDefinition,
params: { urlParams: AutocompleteUrlParams; bodyParams: AutocompleteBodyParams }
): AutocompleteDefinition => {
const { urlParams, bodyParams } = params;
if (urlParams && Object.keys(urlParams).length > 0) {
definition.url_params = urlParams;
Expand All @@ -119,15 +79,19 @@ const addParams = (
return definition;
};

const generateDefinition = (endpoint: Endpoint, schema: Schema): Definition => {
const generateDefinition = (
endpoint: SpecificationTypes.Endpoint,
schema: SpecificationTypes.Model
): AutocompleteDefinition => {
const methods = generateMethods(endpoint);
const patterns = generatePatterns(endpoint);
const documentation = generateDocumentation(endpoint);
let definition: Definition = { methods, patterns, documentation };
let definition: AutocompleteDefinition = {};
const params = generateParams(endpoint, schema);
if (params) {
definition = addParams(definition, params);
}
definition = { ...definition, methods, patterns, documentation };

return definition;
};
Expand All @@ -143,15 +107,15 @@ export function generateConsoleDefinitions({
}) {
const pathToSchemaFile = Path.resolve(specsRepo, 'output/schema/schema.json');
log.info('loading the ES specification schema file');
const schema = JSON.parse(fs.readFileSync(pathToSchemaFile, 'utf8')) as Schema;
const schema = JSON.parse(fs.readFileSync(pathToSchemaFile, 'utf8')) as SpecificationTypes.Model;

const { endpoints } = schema;
log.info(`iterating over endpoints array: ${endpoints.length} endpoints`);
endpoints.forEach((endpoint) => {
const { name } = endpoint;
log.info(name);
const definition = generateDefinition(endpoint, schema);
const fileContent: { [name: string]: Definition } = {
const fileContent: { [name: string]: AutocompleteDefinition } = {
[name]: definition,
};
fs.writeFileSync(
Expand Down
Loading

0 comments on commit ee6ca65

Please sign in to comment.