Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Assistant] Automatically Install Knowledge Base #182763

Merged
merged 40 commits into from
May 20, 2024
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
2e8b823
Adds feature flag and cleaned up 'feature capabilities' touch points
spong May 1, 2024
97e2c35
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 1, 2024
cbcc5de
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 2, 2024
86e5ccc
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 3, 2024
7d0acb0
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 6, 2024
4e4e1e7
Use _inference API to automatically setup ELSER, and install ingest p…
spong May 6, 2024
8fd6b2b
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 7, 2024
316e8c9
Fix settings tests
spong May 7, 2024
71f4dd0
Switch to using installElasticModel()
spong May 8, 2024
8843e87
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 8, 2024
d2c27ac
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 9, 2024
49a9c5a
Resouce check note update
spong May 9, 2024
512806a
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine May 9, 2024
dbc96e4
Creates 'kb entries' OAS and standardize routes and constants
spong May 9, 2024
26006ff
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 9, 2024
acdc0e7
Plumb through kbDataClient create and add kbDataClient support to EsS…
spong May 10, 2024
cd91732
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 10, 2024
0eac540
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 13, 2024
49986ea
Adds ingest pipeline to index template and fixes initial setup and st…
spong May 13, 2024
7354ffc
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 13, 2024
5b3d172
Fix kb document loading and auto elser setup
spong May 14, 2024
60ed588
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 14, 2024
aa81b12
Fixes multi-step setup issue and wires up kbDataClient retrieval in p…
spong May 15, 2024
31a2719
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 15, 2024
be327c1
[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…
kibanamachine May 15, 2024
2b10c9c
Fix remaining UI controls from old impl, and refactor authenticatedUs…
spong May 15, 2024
684488a
Plumb through isSetupInProgress to status endpoints and migrate them …
spong May 15, 2024
146aafb
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 15, 2024
252b8e8
Fix isSetupInProgress update not propagating
spong May 15, 2024
ddf93a8
Relax id validation from UUID to UUID or NonEmptyString to support ES…
spong May 15, 2024
bef08b7
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 15, 2024
0038901
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 16, 2024
7209e09
Update kb entry naming on naming data client get
spong May 17, 2024
17f633e
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 17, 2024
311b6b2
Comments from PR review
spong May 17, 2024
00468fa
Remaining comments from PR review
spong May 17, 2024
239f834
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 17, 2024
297b9bf
Merge branch 'main' into knowledge-by-default
spong May 20, 2024
a1f2820
Align id schema to NonEmptyString
spong May 20, 2024
c91ea58
Merge branch 'main' of github.com:elastic/kibana into knowledge-by-de…
spong May 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions x-pack/packages/kbn-elastic-assistant-common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
export const ELASTIC_AI_ASSISTANT_INTERNAL_API_VERSION = '1';

export const ELASTIC_AI_ASSISTANT_URL = '/api/elastic_assistant';
export const ELASTIC_AI_ASSISTANT_INTERNAL_URL = '/internal/elastic_assistant';

export const ELASTIC_AI_ASSISTANT_CONVERSATIONS_URL = `${ELASTIC_AI_ASSISTANT_URL}/current_user/conversations`;
export const ELASTIC_AI_ASSISTANT_CONVERSATIONS_URL_BY_ID = `${ELASTIC_AI_ASSISTANT_CONVERSATIONS_URL}/{id}`;
Expand All @@ -23,3 +24,8 @@ export const ELASTIC_AI_ASSISTANT_PROMPTS_URL_FIND = `${ELASTIC_AI_ASSISTANT_PRO
export const ELASTIC_AI_ASSISTANT_ANONYMIZATION_FIELDS_URL = `${ELASTIC_AI_ASSISTANT_URL}/anonymization_fields`;
export const ELASTIC_AI_ASSISTANT_ANONYMIZATION_FIELDS_URL_BULK_ACTION = `${ELASTIC_AI_ASSISTANT_ANONYMIZATION_FIELDS_URL}/_bulk_action`;
export const ELASTIC_AI_ASSISTANT_ANONYMIZATION_FIELDS_URL_FIND = `${ELASTIC_AI_ASSISTANT_ANONYMIZATION_FIELDS_URL}/_find`;

// TODO: Update existing 'status' endpoint to take resource as query param as to not conflict with 'entries'
export const ELASTIC_AI_ASSISTANT_KNOWLEDGE_BASE_URL = `${ELASTIC_AI_ASSISTANT_INTERNAL_URL}/knowledge_base/{resource?}`;
export const ELASTIC_AI_ASSISTANT_KNOWLEDGE_BASE_ENTRIES_URL = `${ELASTIC_AI_ASSISTANT_URL}/knowledge_base/entries`;
export const ELASTIC_AI_ASSISTANT_KNOWLEDGE_BASE_ENTRIES_URL_BULK_ACTION = `${ELASTIC_AI_ASSISTANT_URL}/knowledge_base/_bulk_action`;
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ export type AssistantFeatures = { [K in keyof typeof defaultAssistantFeatures]:
* Default features available to the elastic assistant
*/
export const defaultAssistantFeatures = Object.freeze({
assistantKnowledgeBaseByDefault: false,
assistantModelEvaluation: false,
});
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ import { z } from 'zod';
* version: 1
*/

import { UUID, Replacements } from '../conversations/common_attributes.gen';
import { UUID, NonEmptyString } from '../common_attributes.gen';
import { Replacements } from '../conversations/common_attributes.gen';

export type ExecuteConnectorRequestParams = z.infer<typeof ExecuteConnectorRequestParams>;
export const ExecuteConnectorRequestParams = z.object({
Expand All @@ -29,7 +30,7 @@ export type ExecuteConnectorRequestParamsInput = z.input<typeof ExecuteConnector

export type ExecuteConnectorRequestBody = z.infer<typeof ExecuteConnectorRequestBody>;
export const ExecuteConnectorRequestBody = z.object({
conversationId: UUID.optional(),
conversationId: z.union([UUID, NonEmptyString]).optional(),
message: z.string().optional(),
model: z.string().optional(),
subAction: z.enum(['invokeAI', 'invokeStream']),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ paths:
- subAction
properties:
conversationId:
$ref: '../conversations/common_attributes.schema.yaml#/components/schemas/UUID'
oneOf:
- $ref: '../common_attributes.schema.yaml#/components/schemas/UUID'
- $ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
message:
type: string
model:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ import { z } from 'zod';
* version: 2023-10-31
*/

import { UUID, NonEmptyString } from '../conversations/common_attributes.gen';
import { UUID, NonEmptyString } from '../common_attributes.gen';

export type BulkActionSkipReason = z.infer<typeof BulkActionSkipReason>;
export const BulkActionSkipReason = z.literal('ANONYMIZATION_FIELD_NOT_MODIFIED');
Expand Down Expand Up @@ -44,7 +44,7 @@ export const NormalizedAnonymizationFieldError = z.object({

export type AnonymizationFieldResponse = z.infer<typeof AnonymizationFieldResponse>;
export const AnonymizationFieldResponse = z.object({
id: UUID,
id: z.union([UUID, NonEmptyString]),
timestamp: NonEmptyString.optional(),
field: z.string(),
allowed: z.boolean().optional(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -103,9 +103,11 @@ components:
- field
properties:
id:
$ref: '../conversations/common_attributes.schema.yaml#/components/schemas/UUID'
oneOf:
- $ref: '../common_attributes.schema.yaml#/components/schemas/UUID'
- $ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
'timestamp':
$ref: '../conversations/common_attributes.schema.yaml#/components/schemas/NonEmptyString'
$ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
field:
type: string
allowed:
Expand Down Expand Up @@ -232,4 +234,3 @@ components:
type: boolean
anonymized:
type: boolean

Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,6 @@ import { z } from 'zod';

export type GetCapabilitiesResponse = z.infer<typeof GetCapabilitiesResponse>;
export const GetCapabilitiesResponse = z.object({
assistantKnowledgeBaseByDefault: z.boolean(),
assistantModelEvaluation: z.boolean(),
});
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,12 @@ paths:
schema:
type: object
properties:
assistantKnowledgeBaseByDefault:
type: boolean
assistantModelEvaluation:
type: boolean
required:
- assistantKnowledgeBaseByDefault
- assistantModelEvaluation
'400':
description: Generic Error
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { z } from 'zod';

/*
* NOTICE: Do not edit this file manually.
* This file is automatically generated by the OpenAPI Generator, @kbn/openapi-generator.
*
* info:
* title: Common Elastic AI Assistant Attributes
* version: not applicable
*/

/**
* A string that is not empty and does not contain only whitespace
*/
export type NonEmptyString = z.infer<typeof NonEmptyString>;
export const NonEmptyString = z
.string()
.min(1)
.regex(/^(?! *$).+$/);

/**
* A universally unique identifier
*/
export type UUID = z.infer<typeof UUID>;
export const UUID = z.string().uuid();

/**
* Could be any string, not necessarily a UUID
*/
export type User = z.infer<typeof User>;
export const User = z.object({
/**
* User id
*/
id: z.string().optional(),
/**
* User name
*/
name: z.string().optional(),
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
openapi: 3.0.0
info:
title: Common Elastic AI Assistant Attributes
version: 'not applicable'
paths: {}
components:
x-codegen-enabled: true
schemas:
NonEmptyString:
type: string
pattern: ^(?! *$).+$
minLength: 1
description: A string that is not empty and does not contain only whitespace

UUID:
type: string
format: uuid
description: A universally unique identifier

User:
type: object
description: Could be any string, not necessarily a UUID
properties:
id:
type: string
description: User id
name:
type: string
description: User name

Original file line number Diff line number Diff line change
Expand Up @@ -16,35 +16,7 @@ import { z } from 'zod';
* version: not applicable
*/

/**
* A string that is not empty and does not contain only whitespace
*/
export type NonEmptyString = z.infer<typeof NonEmptyString>;
export const NonEmptyString = z
.string()
.min(1)
.regex(/^(?! *$).+$/);

/**
* A universally unique identifier
*/
export type UUID = z.infer<typeof UUID>;
export const UUID = z.string().uuid();

/**
* Could be any string, not necessarily a UUID
*/
export type User = z.infer<typeof User>;
export const User = z.object({
/**
* User id.
*/
id: z.string().optional(),
/**
* User name.
*/
name: z.string().optional(),
});
import { NonEmptyString, UUID, User } from '../common_attributes.gen';

/**
* trace Data
Expand Down Expand Up @@ -180,7 +152,7 @@ export const ConversationSummary = z.object({
export type ErrorSchema = z.infer<typeof ErrorSchema>;
export const ErrorSchema = z
.object({
id: UUID.optional(),
id: z.union([UUID, NonEmptyString]).optional(),
error: z.object({
status_code: z.number().int().min(400),
message: z.string(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,27 +6,6 @@ paths: {}
components:
x-codegen-enabled: true
schemas:
NonEmptyString:
type: string
pattern: ^(?! *$).+$
minLength: 1
description: A string that is not empty and does not contain only whitespace

UUID:
type: string
format: uuid
description: A universally unique identifier

User:
type: object
description: Could be any string, not necessarily a UUID
properties:
id:
type: string
description: User id.
name:
type: string
description: User name.

TraceData:
type: object
Expand Down Expand Up @@ -97,7 +76,7 @@ components:
$ref: '#/components/schemas/MessageRole'
description: Message role.
timestamp:
$ref: '#/components/schemas/NonEmptyString'
$ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
description: The timestamp message was sent or received.
isError:
type: boolean
Expand Down Expand Up @@ -135,7 +114,7 @@ components:
type: string
description: Summary text of the conversation over time.
timestamp:
$ref: '#/components/schemas/NonEmptyString'
$ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
description: The timestamp summary was updated.
public:
type: boolean
Expand All @@ -151,7 +130,9 @@ components:
additionalProperties: false
properties:
id:
$ref: '#/components/schemas/UUID'
oneOf:
- $ref: '../common_attributes.schema.yaml#/components/schemas/UUID'
- $ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
error:
type: object
required:
Expand All @@ -176,8 +157,8 @@ components:
properties:
id:
oneOf:
- $ref: '#/components/schemas/UUID'
- $ref: '#/components/schemas/NonEmptyString'
- $ref: '../common_attributes.schema.yaml#/components/schemas/UUID'
- $ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
title:
type: string
description: The conversation title.
Expand All @@ -187,7 +168,7 @@ components:
summary:
$ref: '#/components/schemas/ConversationSummary'
'timestamp':
$ref: '#/components/schemas/NonEmptyString'
$ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
updatedAt:
description: The last time conversation was updated.
type: string
Expand All @@ -199,7 +180,7 @@ components:
users:
type: array
items:
$ref: '#/components/schemas/User'
$ref: '../common_attributes.schema.yaml#/components/schemas/User'
messages:
type: array
items:
Expand All @@ -225,8 +206,8 @@ components:
properties:
id:
oneOf:
- $ref: '#/components/schemas/UUID'
- $ref: '#/components/schemas/NonEmptyString'
- $ref: '../common_attributes.schema.yaml#/components/schemas/UUID'
- $ref: '../common_attributes.schema.yaml#/components/schemas/NonEmptyString'
title:
type: string
description: The conversation title.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ import { z } from 'zod';
import {
ConversationCreateProps,
ConversationResponse,
UUID,
ConversationUpdateProps,
ConversationMessageCreateProps,
} from './common_attributes.gen';
import { UUID, NonEmptyString } from '../common_attributes.gen';

export type AppendConversationMessageRequestParams = z.infer<
typeof AppendConversationMessageRequestParams
Expand All @@ -31,7 +31,7 @@ export const AppendConversationMessageRequestParams = z.object({
/**
* The conversation's `id` value.
*/
id: UUID,
id: z.union([UUID, NonEmptyString]),
});
export type AppendConversationMessageRequestParamsInput = z.input<
typeof AppendConversationMessageRequestParams
Expand Down Expand Up @@ -60,7 +60,7 @@ export const DeleteConversationRequestParams = z.object({
/**
* The conversation's `id` value.
*/
id: UUID,
id: z.union([UUID, NonEmptyString]),
});
export type DeleteConversationRequestParamsInput = z.input<typeof DeleteConversationRequestParams>;

Expand All @@ -72,7 +72,7 @@ export const ReadConversationRequestParams = z.object({
/**
* The conversation's `id` value.
*/
id: UUID,
id: z.union([UUID, NonEmptyString]),
});
export type ReadConversationRequestParamsInput = z.input<typeof ReadConversationRequestParams>;

Expand All @@ -84,7 +84,7 @@ export const UpdateConversationRequestParams = z.object({
/**
* The conversation's `id` value.
*/
id: UUID,
id: z.union([UUID, NonEmptyString]),
});
export type UpdateConversationRequestParamsInput = z.input<typeof UpdateConversationRequestParams>;

Expand Down
Loading