Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Reporting/Config] allow string representations of duration settings #74202

Merged
merged 41 commits into from
Sep 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
6ecd79f
[Reporting/Config] use better schema methods
tsullivan Aug 3, 2020
0472d45
add createMockConfig
tsullivan Aug 17, 2020
a432276
update documentation
tsullivan Aug 19, 2020
ff00738
fix observable.test
tsullivan Aug 19, 2020
ce6eccb
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 20, 2020
a40626b
add docs links to common options page
tsullivan Aug 20, 2020
88af5e2
make the schema match the docs
tsullivan Aug 20, 2020
2fd26c5
Merge branch 'master' into reporting/config-use-duration
tsullivan Aug 20, 2020
4273afc
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 20, 2020
2157889
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 21, 2020
f35ce37
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 24, 2020
43f5ce0
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 27, 2020
477ce75
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 28, 2020
618ae45
Merge branch 'master' into reporting/config-use-duration
elasticmachine Aug 31, 2020
7aecbdc
Merge branch 'master' into reporting/config-use-duration
tsullivan Aug 31, 2020
7b221f5
wording edits per feedback
tsullivan Aug 31, 2020
7afa565
self edits
tsullivan Aug 31, 2020
1d6f4f9
Merge branch 'master' into reporting/config-use-duration
elasticmachine Sep 3, 2020
05debe2
todo comment
tsullivan Sep 3, 2020
2bb2566
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 9, 2020
556c71c
Merge commit '2bb25663528' into reporting/config-use-duration
tsullivan Sep 9, 2020
0d0f298
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 9, 2020
a2fab88
fix tests
tsullivan Sep 10, 2020
7515e09
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 15, 2020
bfcf1d8
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 16, 2020
5319844
feedback change 1
tsullivan Sep 16, 2020
2fdaf88
schema utils
tsullivan Sep 16, 2020
d9bed34
fix goof
tsullivan Sep 16, 2020
89c0b89
use objects for the defaults when available
tsullivan Sep 16, 2020
d06391e
fix pollInterval
tsullivan Sep 16, 2020
5bc26d9
fix snapshots
tsullivan Sep 16, 2020
79aef77
Update report_listing.tsx
tsullivan Sep 16, 2020
5fdcdea
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 16, 2020
e5f6a51
call new ByteSizeValue on server side only
tsullivan Sep 17, 2020
729d1e7
revert xpack.reporting.poll
tsullivan Sep 17, 2020
a5138e4
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 17, 2020
494fd05
Merge branch 'master' into reporting/config-use-duration
tsullivan Sep 17, 2020
3134289
fix ts
tsullivan Sep 17, 2020
a09f956
fix snapshot
tsullivan Sep 17, 2020
e78fae2
use correct input for duration
tsullivan Sep 17, 2020
7956662
revert reorganize imports
tsullivan Sep 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 26 additions & 28 deletions docs/settings/reporting-settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,14 @@ security is enabled, `xpack.security.encryptionKey`.
[cols="2*<"]
|===
| `xpack.reporting.queue.pollInterval`
| Specifies the number of milliseconds that the reporting poller waits between polling the
index for any pending Reporting jobs. Defaults to `3000` (3 seconds).
| Specify the {ref}/common-options.html#time-units[time] that the reporting poller waits between polling the index for any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to give a brief description on the order of these timeouts, so folks know what to expect with regards to the lifecycle of how these timeouts.

pending Reporting jobs. Can be specified as number of milliseconds. Defaults to `3s`.

| [[xpack-reporting-q-timeout]] `xpack.reporting.queue.timeout` {ess-icon}
| How long each worker has to produce a report. If your machine is slow or under
heavy load, you might need to increase this timeout. Specified in milliseconds.
If a Reporting job execution time goes over this time limit, the job will be
marked as a failure and there will not be a download available.
Defaults to `120000` (two minutes).
| {ref}/common-options.html#time-units[How long] each worker has to produce a report. If your machine is slow or under heavy
load, you might need to increase this timeout. If a Reporting job execution goes over this time limit, the job is marked as a
failure and no download will be available. Can be specified as number of milliseconds.
Defaults to `2m`.

|===

Expand All @@ -127,24 +126,24 @@ control the capturing process.
|===
a| `xpack.reporting.capture.timeouts`
`.openUrl` {ess-icon}
| Specify how long to allow the Reporting browser to wait for the "Loading..." screen
to dismiss and find the initial data for the Kibana page. If the time is
exceeded, a page screenshot is captured showing the current state, and the download link shows a warning message.
Defaults to `60000` (1 minute).
| Specify the {ref}/common-options.html#time-units[time] to allow the Reporting browser to wait for the "Loading..." screen
to dismiss and find the initial data for the page. If the time is exceeded, a screenshot is captured showing the current
page, and the download link shows a warning message. Can be specified as number of milliseconds.
Defaults to `1m`.

a| `xpack.reporting.capture.timeouts`
`.waitForElements` {ess-icon}
| Specify how long to allow the Reporting browser to wait for all visualization
panels to load on the Kibana page. If the time is exceeded, a page screenshot
is captured showing the current state, and the download link shows a warning message. Defaults to `30000` (30
seconds).
| Specify the {ref}/common-options.html#time-units[time] to allow the Reporting browser to wait for all visualization panels
to load on the page. If the time is exceeded, a screenshot is captured showing the current page, and the download link shows
a warning message. Can be specified as number of milliseconds.
Defaults to `30s`.

a| `xpack.reporting.capture.timeouts`
`.renderComplete` {ess-icon}
| Specify how long to allow the Reporting browser to wait for all visualizations to
fetch and render the data. If the time is exceeded, a
page screenshot is captured showing the current state, and the download link shows a warning message. Defaults to
`30000` (30 seconds).
| Specify the {ref}/common-options.html#time-units[time] to allow the Reporting browser to wait for all visualizations to
fetch and render the data. If the time is exceeded, a screenshot is captured showing the current page, and the download link shows a
warning message. Can be specified as number of milliseconds.
Defaults to `30s`.

|===

Expand All @@ -163,11 +162,10 @@ available, but there will likely be errors in the visualizations in the report.
job, as many times as this setting. Defaults to `3`.

| `xpack.reporting.capture.loadDelay`
| When visualizations are not evented, this is the amount of time before
taking a screenshot. All visualizations that ship with {kib} are evented, so this
setting should not have much effect. If you are seeing empty images instead of
visualizations, try increasing this value.
Defaults to `3000` (3 seconds).
| Specify the {ref}/common-options.html#time-units[amount of time] before taking a screenshot when visualizations are not evented.
All visualizations that ship with {kib} are evented, so this setting should not have much effect. If you are seeing empty images
instead of visualizations, try increasing this value.
Defaults to `3s`.

| [[xpack-reporting-browser]] `xpack.reporting.capture.browser.type` {ess-icon}
| Specifies the browser to use to capture screenshots. This setting exists for
Expand Down Expand Up @@ -213,17 +211,17 @@ a| `xpack.reporting.capture.browser`
[cols="2*<"]
|===
| [[xpack-reporting-csv]] `xpack.reporting.csv.maxSizeBytes` {ess-icon}
| The maximum size of a CSV file before being truncated. This setting exists to prevent
large exports from causing performance and storage issues.
Defaults to `10485760` (10mB).
| The maximum {ref}/common-options.html#byte-units[byte size] of a CSV file before being truncated. This setting exists to
prevent large exports from causing performance and storage issues. Can be specified as number of bytes.
Defaults to `10mb`.

| `xpack.reporting.csv.scroll.size`
| Number of documents retrieved from {es} for each scroll iteration during a CSV
export.
Defaults to `500`.

| `xpack.reporting.csv.scroll.duration`
| Amount of time allowed before {kib} cleans the scroll context during a CSV export.
| Amount of {ref}/common-options.html#time-units[time] allowed before {kib} cleans the scroll context during a CSV export.
Defaults to `30s`.

| `xpack.reporting.csv.checkForFormulas`
Expand Down
28 changes: 28 additions & 0 deletions x-pack/plugins/reporting/common/schema_utils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
tsullivan marked this conversation as resolved.
Show resolved Hide resolved
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { ByteSizeValue } from '@kbn/config-schema';
import moment from 'moment';

/*
* For cleaner code: use these functions when a config schema value could be
* one type or another. This allows you to treat the value as one type.
*/

export const durationToNumber = (value: number | moment.Duration): number => {
if (typeof value === 'number') {
return value;
}
return value.asMilliseconds();
};

export const byteSizeValueToNumber = (value: number | ByteSizeValue) => {
if (typeof value === 'number') {
return value;
}

return value.getValueInBytes();
};
11 changes: 7 additions & 4 deletions x-pack/plugins/reporting/public/components/report_listing.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@

import {
EuiBasicTable,
EuiFlexItem,
EuiFlexGroup,
EuiFlexItem,
EuiPageContent,
EuiSpacer,
EuiText,
Expand All @@ -23,6 +23,7 @@ import { Subscription } from 'rxjs';
import { ApplicationStart, ToastsSetup } from 'src/core/public';
import { ILicense, LicensingPluginSetup } from '../../../licensing/public';
import { Poller } from '../../common/poller';
import { durationToNumber } from '../../common/schema_utils';
import { JobStatuses } from '../../constants';
import { checkLicense } from '../lib/license_check';
import { JobQueueEntry, ReportingAPIClient } from '../lib/reporting_api_client';
Expand Down Expand Up @@ -183,17 +184,19 @@ class ReportListingUi extends Component<Props, State> {

public componentDidMount() {
this.mounted = true;
const { pollConfig, license$ } = this.props;
const pollFrequencyInMillis = durationToNumber(pollConfig.jobsRefresh.interval);
this.poller = new Poller({
functionToPoll: () => {
return this.fetchJobs();
},
pollFrequencyInMillis: this.props.pollConfig.jobsRefresh.interval,
pollFrequencyInMillis,
trailing: false,
continuePollingOnError: true,
pollFrequencyErrorMultiplier: this.props.pollConfig.jobsRefresh.intervalErrorMultiplier,
pollFrequencyErrorMultiplier: pollConfig.jobsRefresh.intervalErrorMultiplier,
});
this.poller.start();
this.licenseSubscription = this.props.license$.subscribe(this.licenseHandler);
this.licenseSubscription = license$.subscribe(this.licenseHandler);
}

private licenseHandler = (license: ILicense) => {
Expand Down
4 changes: 2 additions & 2 deletions x-pack/plugins/reporting/public/plugin.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import {
import { ManagementSetup } from '../../../../src/plugins/management/public';
import { SharePluginSetup } from '../../../../src/plugins/share/public';
import { LicensingPluginSetup } from '../../licensing/public';
import { durationToNumber } from '../common/schema_utils';
import { JobId, JobStatusBuckets, ReportingConfigType } from '../common/types';
import { JOB_COMPLETION_NOTIFICATIONS_SESSION_KEY } from '../constants';
import { getGeneralErrorToast } from './components';
Expand Down Expand Up @@ -158,8 +159,7 @@ export class ReportingPublicPlugin implements Plugin<void, void> {
const { http, notifications } = core;
const apiClient = new ReportingAPIClient(http);
const streamHandler = new StreamHandler(notifications, apiClient);
const { interval } = this.config.poll.jobsRefresh;

const interval = durationToNumber(this.config.poll.jobsRefresh.interval);
Rx.timer(0, interval)
.pipe(
takeUntil(this.stop$), // stop the interval when stop method is called
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { InnerSubscriber } from 'rxjs/internal/InnerSubscriber';
import { ignoreElements, map, mergeMap, tap } from 'rxjs/operators';
import { getChromiumDisconnectedError } from '../';
import { BROWSER_TYPE } from '../../../../common/constants';
import { durationToNumber } from '../../../../common/schema_utils';
import { CaptureConfig } from '../../../../server/types';
import { LevelLogger } from '../../../lib';
import { safeChildProcess } from '../../safe_child_process';
Expand Down Expand Up @@ -90,7 +91,7 @@ export class HeadlessChromiumDriverFactory {

// Set the default timeout for all navigation methods to the openUrl timeout (30 seconds)
// All waitFor methods have their own timeout config passed in to them
page.setDefaultTimeout(this.captureConfig.timeouts.openUrl);
page.setDefaultTimeout(durationToNumber(this.captureConfig.timeouts.openUrl));

logger.debug(`Browser page driver created`);
} catch (err) {
Expand Down
2 changes: 2 additions & 0 deletions x-pack/plugins/reporting/server/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ export const config: PluginConfigDescriptor<ReportingConfigType> = {
unused('capture.concurrency'),
unused('capture.settleTime'),
unused('capture.timeout'),
unused('poll.jobCompletionNotifier.intervalErrorMultiplier'),
unused('poll.jobsRefresh.intervalErrorMultiplier'),
unused('kibanaApp'),
],
};
Loading
Loading