Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Ingest] Add additional attributes to the Datasources Saved Object #66127

Merged
5 changes: 5 additions & 0 deletions x-pack/plugins/endpoint/common/generate_data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -504,10 +504,15 @@ export class EndpointDocGenerator {
* Generates an Ingest `datasource` that includes the Endpoint Policy data
*/
public generatePolicyDatasource(): PolicyData {
const created = new Date(Date.now() - 8.64e7).toISOString(); // 24h ago
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I imagine is just for testing purpose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, test purposes to mock up data,

return {
id: this.seededUUIDv4(),
name: 'Endpoint Policy',
description: 'Policy to protect the worlds data',
created_on: created,
created_by: 'elastic',
updated_on: new Date().toISOString(),
updated_by: 'elastic',
config_id: this.seededUUIDv4(),
enabled: true,
output_id: '',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ describe('policy details: ', () => {
id: '',
name: '',
description: '',
created_on: '',
created_by: '',
updated_on: '',
updated_by: '',
config_id: '',
enabled: true,
output_id: '',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ describe('Ingest Manager - storedDatasourceToAgentDatasource', () => {
id: 'some-uuid',
name: 'mock-datasource',
description: '',
created_on: '',
created_by: '',
updated_on: '',
updated_by: '',
config_id: '',
enabled: true,
output_id: '',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,8 @@ export interface Datasource extends Omit<NewDatasource, 'inputs'> {
id: string;
inputs: DatasourceInput[];
revision: number;
updated_on: string;
updated_by: string;
created_on: string;
created_by: string;
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
*/

import { SavedObjectsServiceSetup, SavedObjectsType } from 'kibana/server';
import { EncryptedSavedObjectsPluginSetup } from '../../encrypted_saved_objects/server';
import { EncryptedSavedObjectsPluginSetup } from '../../../encrypted_saved_objects/server';
import {
OUTPUT_SAVED_OBJECT_TYPE,
AGENT_CONFIG_SAVED_OBJECT_TYPE,
Expand All @@ -16,7 +16,8 @@ import {
AGENT_ACTION_SAVED_OBJECT_TYPE,
ENROLLMENT_API_KEYS_SAVED_OBJECT_TYPE,
GLOBAL_SETTINGS_SAVED_OBJET_TYPE,
} from './constants';
} from '../constants';
import { migrateDatasourcesToV790 } from './migrations/datasources_v790';

/*
* Saved object types and mappings
Expand Down Expand Up @@ -218,8 +219,15 @@ const savedObjectTypes: { [key: string]: SavedObjectsType } = {
},
},
revision: { type: 'integer' },
updated_on: { type: 'keyword' },
updated_by: { type: 'keyword' },
created_on: { type: 'keyword' },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using the type date?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point @nchaulet . I copied the definition from the Agent Config SO which had keyword, but now see that Enrollment API Keys SO does use date. I will change it.
Thanks 👍

created_by: { type: 'keyword' },
},
},
migrations: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though I don't expect users to migrate from 7.8 to 7.9 I like that we get into the habit of providing a migration path! 👍

'7.9.0': migrateDatasourcesToV790,
},
},
[PACKAGES_SAVED_OBJECT_TYPE]: {
name: PACKAGES_SAVED_OBJECT_TYPE,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { SavedObjectMigrationFn } from 'kibana/server';
import { cloneDeep } from 'lodash';
import { Datasource } from '../../types/models';

type Pre790Datasource = Exclude<
Datasource,
'created_on' | 'created_by' | 'updated_on' | 'updated_by'
>;

export const migrateDatasourcesToV790: SavedObjectMigrationFn<
Pre790Datasource,
Datasource
> = doc => {
const updatedDatasource = cloneDeep(doc);
const defDate = new Date().toISOString();

updatedDatasource.attributes.created_by = 'system';
updatedDatasource.attributes.created_on = updatedDatasource?.updated_at ?? defDate;
updatedDatasource.attributes.updated_by = 'system';
updatedDatasource.attributes.updated_on = updatedDatasource?.updated_at ?? defDate;

return updatedDatasource;
};
7 changes: 7 additions & 0 deletions x-pack/plugins/ingest_manager/server/services/datasource.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,16 @@ class DatasourceService {
datasource: NewDatasource,
options?: { id?: string; user?: AuthenticatedUser }
): Promise<Datasource> {
const isoDate = new Date().toISOString();
const newSo = await soClient.create<Omit<Datasource, 'id'>>(
SAVED_OBJECT_TYPE,
{
...datasource,
revision: 1,
created_on: isoDate,
created_by: options?.user?.username ?? 'system',
updated_on: isoDate,
updated_by: options?.user?.username ?? 'system',
},
options
);
Expand Down Expand Up @@ -134,6 +139,8 @@ class DatasourceService {
await soClient.update<Datasource>(SAVED_OBJECT_TYPE, id, {
...datasource,
revision: oldDatasource.revision + 1,
updated_on: new Date().toISOString(),
updated_by: options?.user?.username ?? 'system',
});

// Bump revision of associated agent config
Expand Down