Skip to content

Commit

Permalink
Pull request 1736: 4299-querylog-stats-api
Browse files Browse the repository at this point in the history
Merge in DNS/adguard-home from 4299-querylog-stats-api to master

Updates #1717.
Updates #4299.

Squashed commit of the following:

commit 5b706b7
Merge: 48b62b0 306c198
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 22 13:53:09 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit 48b62b0
Author: Vladislav Abdulmyanov <v.abdulmyanov@adguard.com>
Date:   Wed Mar 22 12:25:04 2023 +0200

    client: fix styles, add titles and descrs

commit 97e31cf
Author: Vladislav Abdulmyanov <v.abdulmyanov@adguard.com>
Date:   Tue Mar 21 18:38:12 2023 +0200

    client: add ignored domains for querylog

commit 24d75c4
Author: Vladislav Abdulmyanov <v.abdulmyanov@adguard.com>
Date:   Tue Mar 21 18:21:13 2023 +0200

    client: add ignore domains for stats

commit eefc389
Merge: 978675e 1daabb9
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Mar 21 10:53:35 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit 978675e
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Mar 21 10:53:11 2023 +0300

    openapi: fix chlog

commit 2ed3300
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Mar 20 17:49:07 2023 +0300

    home: fix tests

commit 6af1152
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Mar 20 17:40:16 2023 +0300

    home: fix typo

commit 56acdfd
Merge: 319da34 48431f8
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Mar 20 17:32:58 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit 319da34
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Fri Mar 3 17:34:38 2023 +0300

    querylog: fix docs

commit d5a8f24
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Fri Mar 3 11:42:00 2023 +0300

    stats: fix docs

commit e0cbfc1
Merge: 4743c81 012e5be
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 1 18:45:17 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit 4743c81
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 1 18:14:16 2023 +0300

    all: imp code; fix time conversion

commit 34310cf
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 1 12:34:11 2023 +0300

    chlog: restore order

commit cadd864
Merge: 2f3e25b bb22643
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 1 12:26:06 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit 2f3e25b
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Wed Mar 1 12:25:14 2023 +0300

    all: fix fmt

commit d54022b
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Tue Feb 28 16:16:40 2023 +0300

    all: imp code; fix chlog

commit df22de9
Merge: e1ea479 a772212
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Feb 27 17:24:09 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit e1ea479
Merge: d7db0a5 bb80a7c
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Feb 27 17:23:20 2023 +0300

    Merge branch 'master' into 4299-querylog-stats-api

commit d7db0a5
Author: Stanislav Chzhen <s.chzhen@adguard.com>
Date:   Mon Feb 27 17:12:20 2023 +0300

    all: imp docs

... and 15 more commits
  • Loading branch information
schzhn committed Mar 23, 2023
1 parent 306c198 commit 143616c
Show file tree
Hide file tree
Showing 29 changed files with 994 additions and 156 deletions.
45 changes: 42 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ NOTE: Add new changes BELOW THIS COMMENT.

### Added

- Two new HTTP APIs, `PUT /control/stats/config/update` and `GET
control/stats/config`, which can be used to set and receive the query log
configuration. See openapi/openapi.yaml for the full description.
- Two new HTTP APIs, `PUT /control/querylog/config/update` and `GET
control/querylog/config`, which can be used to set and receive the statistics
configuration. See openapi/openapi.yaml for the full description.
- The ability to set custom IP for EDNS Client Subnet by using the DNS-server
configuration section on the DNS settings page in the UI ([#1472]).
- The ability to manage safesearch for each service by using the new
Expand All @@ -37,8 +43,26 @@ NOTE: Add new changes BELOW THIS COMMENT.

#### Configuration Changes

In this release, the schema version has changed from 17 to 19.
In this release, the schema version has changed from 17 to 20.

- Property `statistics.interval`, which in schema versions 19 and earlier used
to be an integer number of days, is now a string with a human-readable
duration:

```yaml
# BEFORE:
'statistics':
#
'interval': 1

# AFTER:
'statistics':
#
'interval': '24h'
```
To rollback this change, convert the property back into days and change the
`schema_version` back to `19`.
- The `dns.safesearch_enabled` field has been replaced with `safe_search`
object containing per-service settings.
- The `clients.persistent.safesearch_enabled` field has been replaced with
Expand All @@ -64,6 +88,23 @@ In this release, the schema version has changed from 17 to 19.
client's specific `clients.persistent.safesearch` and then change the
`schema_version` back to `17`.

### Deprecated

- The `GET /control/stats_info` HTTP API; use the new `GET
/control/stats/config` API instead.

**NOTE:** If interval is custom then it will be equal to `90` days for
compatibility reasons. See openapi/openapi.yaml and `openapi/CHANGELOG.md`.
- The `POST /control/stats_config` HTTP API; use the new `PUT
/control/stats/config/update` API instead.
- The `GET /control/querylog_info` HTTP API; use the new `GET
/control/querylog/config` API instead.

**NOTE:** If interval is custom then it will be equal to `90` days for
compatibility reasons. See openapi/openapi.yaml and `openapi/CHANGELOG.md`.
- The `POST /control/querylog_config` HTTP API; use the new `PUT
/control/querylog/config/update` API instead.

### Fixed

- Panic caused by empty top-level domain name label in `/etc/hosts` files
Expand Down Expand Up @@ -103,8 +144,6 @@ See also the [v0.107.26 GitHub milestone][ms-v0.107.26].

#### Configuration Changes

In this release, the schema version has changed from 16 to 17.

- Property `edns_client_subnet`, which in schema versions 16 and earlier used
to be a part of the `dns` object, is now part of the `dns.edns_client_subnet`
object:
Expand Down
4 changes: 4 additions & 0 deletions client/src/__locales/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,10 @@
"statistics_retention_confirm": "Are you sure you want to change statistics retention? If you decrease the interval value, some data will be lost",
"statistics_cleared": "Statistics successfully cleared",
"statistics_enable": "Enable statistics",
"ignore_domains": "Ignored domains (separated by newline)",
"ignore_domains_title": "Ignored domains",
"ignore_domains_desc_stats": "Queries for these domains are not written to the statistics",
"ignore_domains_desc_query": "Queries for these domains are not written to the query log",
"interval_hours": "{{count}} hour",
"interval_hours_plural": "{{count}} hours",
"filters_configuration": "Filters configuration",
Expand Down
2 changes: 1 addition & 1 deletion client/src/actions/queryLogs.js
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ export const getLogsConfigSuccess = createAction('GET_LOGS_CONFIG_SUCCESS');
export const getLogsConfig = () => async (dispatch) => {
dispatch(getLogsConfigRequest());
try {
const data = await apiClient.getQueryLogInfo();
const data = await apiClient.getQueryLogConfig();
dispatch(getLogsConfigSuccess(data));
} catch (error) {
dispatch(addErrorToast({ error }));
Expand Down
2 changes: 1 addition & 1 deletion client/src/actions/stats.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ export const getStatsConfigSuccess = createAction('GET_STATS_CONFIG_SUCCESS');
export const getStatsConfig = () => async (dispatch) => {
dispatch(getStatsConfigRequest());
try {
const data = await apiClient.getStatsInfo();
const data = await apiClient.getStatsConfig();
dispatch(getStatsConfigSuccess(data));
} catch (error) {
dispatch(addErrorToast({ error }));
Expand Down
20 changes: 10 additions & 10 deletions client/src/api/Api.js
Original file line number Diff line number Diff line change
Expand Up @@ -497,9 +497,9 @@ class Api {
// Settings for statistics
GET_STATS = { path: 'stats', method: 'GET' };

STATS_INFO = { path: 'stats_info', method: 'GET' };
GET_STATS_CONFIG = { path: 'stats/config', method: 'GET' };

STATS_CONFIG = { path: 'stats_config', method: 'POST' };
UPDATE_STATS_CONFIG = { path: 'stats/config/update', method: 'PUT' };

STATS_RESET = { path: 'stats_reset', method: 'POST' };

Expand All @@ -508,13 +508,13 @@ class Api {
return this.makeRequest(path, method);
}

getStatsInfo() {
const { path, method } = this.STATS_INFO;
getStatsConfig() {
const { path, method } = this.GET_STATS_CONFIG;
return this.makeRequest(path, method);
}

setStatsConfig(data) {
const { path, method } = this.STATS_CONFIG;
const { path, method } = this.UPDATE_STATS_CONFIG;
const config = {
data,
};
Expand All @@ -529,9 +529,9 @@ class Api {
// Query log
GET_QUERY_LOG = { path: 'querylog', method: 'GET' };

QUERY_LOG_CONFIG = { path: 'querylog_config', method: 'POST' };
UPDATE_QUERY_LOG_CONFIG = { path: 'querylog/config/update', method: 'PUT' };

QUERY_LOG_INFO = { path: 'querylog_info', method: 'GET' };
GET_QUERY_LOG_CONFIG = { path: 'querylog/config', method: 'GET' };

QUERY_LOG_CLEAR = { path: 'querylog_clear', method: 'POST' };

Expand All @@ -543,13 +543,13 @@ class Api {
return this.makeRequest(url, method);
}

getQueryLogInfo() {
const { path, method } = this.QUERY_LOG_INFO;
getQueryLogConfig() {
const { path, method } = this.GET_QUERY_LOG_CONFIG;
return this.makeRequest(path, method);
}

setQueryLogConfig(data) {
const { path, method } = this.QUERY_LOG_CONFIG;
const { path, method } = this.UPDATE_QUERY_LOG_CONFIG;
const config = {
data,
};
Expand Down
36 changes: 31 additions & 5 deletions client/src/components/Settings/LogsConfig/Form.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,28 @@ import { Field, reduxForm } from 'redux-form';
import { Trans, withTranslation } from 'react-i18next';
import flow from 'lodash/flow';

import { CheckboxField, renderRadioField, toFloatNumber } from '../../../helpers/form';
import { FORM_NAME, QUERY_LOG_INTERVALS_DAYS } from '../../../helpers/constants';
import {
CheckboxField,
renderRadioField,
toFloatNumber,
renderTextareaField,
} from '../../../helpers/form';
import {
FORM_NAME,
QUERY_LOG_INTERVALS_DAYS,
HOUR,
DAY,
} from '../../../helpers/constants';
import '../FormButton.css';

const getIntervalTitle = (interval, t) => {
switch (interval) {
case 0.25:
case 6 * HOUR:
return t('interval_6_hour');
case 1:
case DAY:
return t('interval_24_hour');
default:
return t('interval_days', { count: interval });
return t('interval_days', { count: interval / DAY });
}
};

Expand Down Expand Up @@ -66,6 +76,22 @@ const Form = (props) => {
{getIntervalFields(processing, t, toFloatNumber)}
</div>
</div>
<label className="form__label form__label--with-desc">
<Trans>ignore_domains_title</Trans>
</label>
<div className="form__desc form__desc--top">
<Trans>ignore_domains_desc_query</Trans>
</div>
<div className="form__group form__group--settings">
<Field
name="ignored"
type="textarea"
className="form-control form-control--textarea font-monospace text-input"
component={renderTextareaField}
placeholder={t('ignore_domains')}
disabled={processing}
/>
</div>
<div className="mt-5">
<button
type="submit"
Expand Down
10 changes: 7 additions & 3 deletions client/src/components/Settings/LogsConfig/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ class LogsConfig extends Component {
const { t, interval: prevInterval } = this.props;
const { interval } = values;

const data = { ...values, ignored: values.ignored ? values.ignored.split('\n') : [] };

if (interval !== prevInterval) {
// eslint-disable-next-line no-alert
if (window.confirm(t('query_log_retention_confirm'))) {
this.props.setLogsConfig(values);
this.props.setLogsConfig(data);
}
} else {
this.props.setLogsConfig(values);
this.props.setLogsConfig(data);
}
};

Expand All @@ -30,7 +32,7 @@ class LogsConfig extends Component {

render() {
const {
t, enabled, interval, processing, processingClear, anonymize_client_ip,
t, enabled, interval, processing, processingClear, anonymize_client_ip, ignored,
} = this.props;

return (
Expand All @@ -45,6 +47,7 @@ class LogsConfig extends Component {
enabled,
interval,
anonymize_client_ip,
ignored: ignored.join('\n'),
}}
onSubmit={this.handleFormSubmit}
processing={processing}
Expand All @@ -62,6 +65,7 @@ LogsConfig.propTypes = {
enabled: PropTypes.bool.isRequired,
anonymize_client_ip: PropTypes.bool.isRequired,
processing: PropTypes.bool.isRequired,
ignored: PropTypes.array.isRequired,
processingClear: PropTypes.bool.isRequired,
setLogsConfig: PropTypes.func.isRequired,
clearLogs: PropTypes.func.isRequired,
Expand Down
48 changes: 30 additions & 18 deletions client/src/components/Settings/StatsConfig/Form.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,31 @@ import { Field, reduxForm } from 'redux-form';
import { Trans, withTranslation } from 'react-i18next';
import flow from 'lodash/flow';

import { renderRadioField, toNumber, CheckboxField } from '../../../helpers/form';
import { FORM_NAME, STATS_INTERVALS_DAYS, DISABLED_STATS_INTERVAL } from '../../../helpers/constants';
import {
renderRadioField,
toNumber,
CheckboxField,
renderTextareaField,
} from '../../../helpers/form';
import {
FORM_NAME,
STATS_INTERVALS_DAYS,
DAY,
} from '../../../helpers/constants';
import '../FormButton.css';

const getIntervalTitle = (interval, t) => {
switch (interval) {
const getIntervalTitle = (intervalMs, t) => {
switch (intervalMs / DAY) {
case 1:
return t('interval_24_hour');
default:
return t('interval_days', { count: interval });
return t('interval_days', { count: intervalMs / DAY });
}
};

const Form = (props) => {
const {
handleSubmit,
change,
processing,
submitting,
invalid,
Expand All @@ -38,13 +46,6 @@ const Form = (props) => {
component={CheckboxField}
placeholder={t('statistics_enable')}
disabled={processing}
onChange={(event) => {
if (event.target.checked) {
change('interval', STATS_INTERVALS_DAYS[0]);
} else {
change('interval', DISABLED_STATS_INTERVAL);
}
}}
/>
</div>
<label className="form__label form__label--with-desc">
Expand All @@ -65,15 +66,26 @@ const Form = (props) => {
placeholder={getIntervalTitle(interval, t)}
normalize={toNumber}
disabled={processing}
onChange={(event) => {
if (event.target.checked) {
change('enabled', true);
}
}}
/>
))}
</div>
</div>
<label className="form__label form__label--with-desc">
<Trans>ignore_domains_title</Trans>
</label>
<div className="form__desc form__desc--top">
<Trans>ignore_domains_desc_stats</Trans>
</div>
<div className="form__group form__group--settings">
<Field
name="ignored"
type="textarea"
className="form-control form-control--textarea font-monospace text-input"
component={renderTextareaField}
placeholder={t('ignore_domains')}
disabled={processing}
/>
</div>
<div className="mt-5">
<button
type="submit"
Expand Down
15 changes: 11 additions & 4 deletions client/src/components/Settings/StatsConfig/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ import Card from '../../ui/Card';
import Form from './Form';

class StatsConfig extends Component {
handleFormSubmit = (values) => {
handleFormSubmit = ({ enabled, interval, ignored }) => {
const { t, interval: prevInterval } = this.props;
const config = { interval: values.interval };
const config = {
enabled,
interval,
ignored: ignored ? ignored.split('\n') : [],
};

if (config.interval < prevInterval) {
if (window.confirm(t('statistics_retention_confirm'))) {
Expand All @@ -29,7 +33,7 @@ class StatsConfig extends Component {

render() {
const {
t, interval, processing, processingReset,
t, interval, processing, processingReset, ignored, enabled,
} = this.props;

return (
Expand All @@ -42,7 +46,8 @@ class StatsConfig extends Component {
<Form
initialValues={{
interval,
enabled: !!interval,
enabled,
ignored: ignored.join('\n'),
}}
onSubmit={this.handleFormSubmit}
processing={processing}
Expand All @@ -57,6 +62,8 @@ class StatsConfig extends Component {

StatsConfig.propTypes = {
interval: PropTypes.number.isRequired,
ignored: PropTypes.array.isRequired,
enabled: PropTypes.bool.isRequired,
processing: PropTypes.bool.isRequired,
processingReset: PropTypes.bool.isRequired,
setStatsConfig: PropTypes.func.isRequired,
Expand Down
Loading

0 comments on commit 143616c

Please sign in to comment.