Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Create Maps section and move pages #2424

Closed
wants to merge 40 commits into from
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
9a618a5
Create Maps section and move pages
vagimeli Jan 18, 2023
2765900
Edit front matter
vagimeli Jan 18, 2023
d71887c
Updates to Security Analytics documentation (#2408)
cwillum Jan 18, 2023
753b45e
Add GPU acceleration documentation (#2384)
Naarcha-AWS Jan 18, 2023
bd93752
Adds cluster health by awareness attribute documentation (#2398)
kolchfa-aws Jan 18, 2023
c441bef
Updates remote-backed storage documentation (#2363)
kolchfa-aws Jan 18, 2023
7f6096d
Move pages (#2425)
vagimeli Jan 18, 2023
8026b6d
Add Query String for rollups (#2428)
Naarcha-AWS Jan 19, 2023
6fb83d8
Adds query string query documentation (#2427)
kolchfa-aws Jan 19, 2023
87859f8
Editorial for Admin UI index operations (#2421)
ariamarble Jan 19, 2023
b3abdbc
Edit front matter and make copy edits to proper name for WMS
vagimeli Jan 19, 2023
b4d7364
Formatting updates
vagimeli Jan 19, 2023
52eb943
Add redirect
vagimeli Jan 19, 2023
87bf387
Refresh documentation
vagimeli Jan 20, 2023
5f7e570
Admin UI further editorial updates (#2444)
ariamarble Jan 19, 2023
e660bd6
Adds second image style with no border (#2445)
kolchfa-aws Jan 20, 2023
ef02ba3
Adds installation guide for OpenSearch Dashboards debian distribution…
Jan 20, 2023
3955777
Update field mapping documentation for Security Analytics (#2422)
cwillum Jan 23, 2023
787fc76
Minor changes to Data Prepper index.md. (#2426)
carolxob Jan 23, 2023
52659f7
Editorial changes (#2454)
Jan 23, 2023
d53c9c6
add two new settings for Anomaly Detection 2.5 (#2450)
amitgalitz Jan 23, 2023
02d7c7d
Add new ML cluster settings for 2.5 (#2442)
Naarcha-AWS Jan 23, 2023
2eba0f2
Remove Install and Configure home page (#2449)
Naarcha-AWS Jan 23, 2023
6138632
Revert "Admin UI further editorial updates (#2444)"
vagimeli Jan 30, 2023
22503d7
Revert "Adds second image style with no border (#2445)"
vagimeli Jan 30, 2023
54011dc
Revert "Update field mapping documentation for Security Analytics (#2…
vagimeli Jan 30, 2023
4c33af8
Revert "Minor changes to Data Prepper index.md. (#2426)"
vagimeli Jan 30, 2023
f7b26f5
Revert "Editorial changes (#2454)"
vagimeli Jan 30, 2023
2478ae2
Revert "add two new settings for Anomaly Detection 2.5 (#2450)"
vagimeli Jan 30, 2023
00aaeba
Revert "Remove Install and Configure home page (#2449)"
vagimeli Jan 30, 2023
ccbd807
Revert "Updates to Security Analytics documentation (#2408)"
vagimeli Jan 30, 2023
e47a6ab
Revert "Add GPU acceleration documentation (#2384)"
vagimeli Jan 30, 2023
9a5356d
Revert "Adds cluster health by awareness attribute documentation (#23…
vagimeli Jan 30, 2023
dfbf529
Revert "Updates remote-backed storage documentation (#2363)"
vagimeli Jan 30, 2023
27fc393
Revert "Add Query String for rollups (#2428)"
vagimeli Jan 30, 2023
643a5d9
Revert "Adds query string query documentation (#2427)"
vagimeli Jan 30, 2023
fa2808b
Revert "Editorial for Admin UI index operations (#2421)"
vagimeli Jan 30, 2023
38402b7
Revert "Move pages (#2425)"
vagimeli Jan 30, 2023
b869d85
Revert "Adds installation guide for OpenSearch Dashboards debian dist…
vagimeli Jan 30, 2023
f6ef40e
Revert "Add new ML cluster settings for 2.5 (#2442)"
vagimeli Jan 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 102 additions & 3 deletions _api-reference/cluster-health.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ The following table lists the available query parameters. All query parameters a
Parameter | Type | Description
:--- | :--- | :---
expand_wildcards | Enum | Expands wildcard expressions to concrete indexes. Combine multiple values with commas. Supported values are `all`, `open`, `closed`, `hidden`, and `none`. Default is `open`.
level | Enum | The level of detail for returned health information. Supported values are `cluster`, `indices`, and `shards`. Default is `cluster`.
level | Enum | The level of detail for returned health information. Supported values are `cluster`, `indices`, `shards`, and `awareness_attributes`. Default is `cluster`.
awareness_attribute | String | The name of the awareness attribute, for which to return cluster health (for example, `zone`). Applicable only if `level` is set to `awareness_attributes`.
local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is false.
cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds.
timeout | Time | The amount of time to wait for a response. If the timeout expires, the request fails. Default is 30 seconds.
Expand Down Expand Up @@ -101,8 +102,106 @@ The following table lists all response fields.
|number_of_in_flight_fetch | Integer | The number of unfinished fetches. |
|task_max_waiting_in_queue_millis | Integer | The maximum wait time for all tasks waiting to be performed, in milliseconds. |
|active_shards_percent_as_number | Double | The percentage of active shards in the cluster. |
|awareness_attributes | Object | Contains cluster health information for each awareness attribute. |

## Returning cluster health by awareness attribute

To check cluster health by awareness attribute (for example, zone or rack), specify `awareness_attributes` in the `level` query parameter:

```json
GET _cluster/health?level=awareness_attributes
```

The response contains cluster health metrics partitioned by awareness attribute:

```json
{
"cluster_name": "runTask",
"status": "green",
"timed_out": false,
"number_of_nodes": 3,
"number_of_data_nodes": 3,
"discovered_master": true,
"discovered_cluster_manager": true,
"active_primary_shards": 0,
"active_shards": 0,
"relocating_shards": 0,
"initializing_shards": 0,
"unassigned_shards": 0,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks": 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 100,
"awareness_attributes": {
"zone": {
"zone-3": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
},
"zone-1": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
},
"zone-2": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
}
},
"rack": {
"rack-3": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
},
"rack-1": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
},
"rack-2": {
"active_shards": 0,
"initializing_shards": 0,
"relocating_shards": 0,
"unassigned_shards": 0,
"data_nodes": 1,
"weight": 1
}
}
}
}
```

If you're interested in a particular awareness attribute, you can include the name of the awareness attribute as a query parameter:

```json
GET _cluster/health?level=awareness_attributes&awareness_attribute=zone
```

In response to the preceding request, OpenSearch returns cluster health information only for the `zone` awareness attribute.

The unassigned shard information will be accurate only if you [enable replica count enforcement]({{site.url}}{{site.baseurl}}/opensearch/cluster#forced-replica-count-enforcement) and [configure forced awareness]({{site.url}}{{site.baseurl}}/opensearch/cluster#forced-awareness) for the awareness attribute either before cluster start or after cluster start but before any indexing requests. If you enable replica enforcement after the cluster receives indexing requests, the unassigned shard information may be inaccurate. If you don't configure replica count enforcement and forced awareness, the `unassigned_shards` field will contain -1.
{: .warning}

## Required permissions

If you use the security plugin, make sure you have the appropriate permissions:
`cluster:monitor/health`.
If you use the Security plugin, make sure you have the appropriate permissions:
`cluster:monitor/health`.
64 changes: 41 additions & 23 deletions _dashboards/admin-ui-index/index-management.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
layout: default
title: Index management
parent: Admin UI for index operations in OpenSearch Dashboards
nav_order: 10
parent: Index and snapshot management in OpenSearch Dashboards
nav_order: 16
---

# Index management
Introduced 2.5
{: .label .label-purple }

The Index Management section in the admin UI allows you to perform the operations available in the [Index API]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/) from OpenSearch Dashboards' web UI.
In the **Index Management** section, you can perform the operations available in the [Index API]({{site.url}}{{site.baseurl}}/api-reference/index-apis/index/).

## Index policies

Expand Down Expand Up @@ -49,9 +49,15 @@ To attach policies to indexes, perform the following steps:

## Indexes

The **Indices** section displays a list of indexes in your OpenSearch cluster. For each index included, you can see its heath status (`green`, `yellow`, or `red`), policy (if the index is managed by a policy), status, total size, size of primaries, total documents, deleted documents, primaries and replicas.
The **Indices** section displays a list of indexes in your OpenSearch cluster. For each index included, you can see its health status (`green`, `yellow`, or `red`), policy (if the index is managed by a policy), status, total size, size of primaries, total documents, deleted documents, primaries, and replicas.

#### Create index
The following are the three index health statuses:

- Green: All primary and replica shards are assigned.
- Yellow: At least one replica shard is not assigned.
- Red: At least one primary shard is not assigned.

### Create index

While you can [create an index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/create-index/) by using a document as a base, you can also create an empty index for later use.

Expand All @@ -66,11 +72,11 @@ You can also add fields and objects using either the visual editor or the JSON e

**Advanced settings** allows you to upload a JSON configuration.

#### Apply policy
### Apply policy

If you analyze time series data, you likely want to prioritize new data over old data. You might periodically perform certain operations on older indexes, such as reducing replica count or deleting them.

[Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) (ISM) is a plugin that lets you automate these periodic administrative operations by triggering them based on changes in the index age, index size, or number of documents. The admin UI uses the ISM plugin to define policies that automatically handle index rollovers or deletions to fit your use case.
[Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) (ISM) is a plugin that lets you automate these periodic administrative operations by triggering them based on changes in the index age, index size, or number of documents. You can define policies that automatically handle index rollovers or deletions to fit your use case.

For example, you can define a policy that moves your index into a **read_only** state after 30 days and then deletes it after a set period of 90 days. You can also set up the policy to send you a notification message when the index is deleted.

Expand All @@ -80,43 +86,43 @@ To apply a policy, select the index to which you want to apply the policy in the

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/apply-policy.PNG" alt="User interface showing apply policy prompt">

#### Close
### Close index

The [close index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/close-index/) operation closes an index. Once an index is closed, you cannot add data to it or search for any data within the index.

To close an index, select the index you want to close in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Close** from the dropdown list.

#### Open
### Open index

The [open index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/open-index/) operation opens a closed index, letting you add or search for data within the index.

To open an index, select the index you want to open in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Open** from the dropdown list.

#### Reindex
### Reindex

The [reindex]({{site.url}}{{site.baseurl}}/api-reference/document-apis/reindex/) operation lets you copy all your data or a subset of data from a source index into a destination index.

To reindex an index, select the index in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Reindex** from the dropdown list.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/reindex-expanded.png" alt="User interface showing reindex prompt">

#### Shrink
### Shrink index

The [shrink]({{site.url}}{{site.baseurl}}/api-reference/index-apis/shrink-index/) index operation moves all of your data in an existing index into a new index with fewer primary shards.
The [shrink]({{site.url}}{{site.baseurl}}/api-reference/index-apis/shrink-index/) index operation copies all of the data in an existing index into a new index with fewer primary shards.

To shrink an index, select the index you want to shrink in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Shrink** from the dropdown list.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/shrink.png" alt="User interface showing shrink prompt">

#### Split
### Split index

The [split index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/split/) operation splits an existing read-only index into a new index, splitting each primary shard into a number of primary shards in the new index.

To split an index, select the index you want to split in the **Indices** list under **Index Management**. Then select the **Actions** button, and select **Split** from the dropdown list.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/split-expanded.png" alt="User interface showing split page">

#### Delete
### Delete index

If you no longer need an index, you can use the [delete index]({{site.url}}{{site.baseurl}}/api-reference/index-apis/delete-index/) operation to delete it.

Expand All @@ -128,22 +134,33 @@ To delete an index, select the index you want to delete in the **Indices** list

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/templates.PNG" alt="User interface showing Templates page">

### Creating a template

To create a template, select the **Create template** button on the **Templates** page under **Index Management**.

Next, define the template:

1. Enter the template name.
2. Select the template type.
3. Specify any index patterns you would like to use.
4. Set the priority of the template.
5. Select an index alias.
6. Set the number of primary shards.
7. Set the number of replicas.
8. Set the refresh intervals.
9. Add fields and objects for your index mapping using either the visual editor or the JSON editor.
1. Select the template type.
1. Specify any index patterns you would like to use.
1. Set the priority of the template.
1. Select an index alias.
1. Set the number of primary shards.
1. Set the number of replicas.
1. Set the refresh intervals.
1. Add fields and objects for your index mapping using either the visual editor or the JSON editor.
1. Under **Advanced Settings** you can specify advanced index settings with a comma-delimited list.

<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/create-template-expanded.png" alt="User interface showing Create Template page">

### Editing a template

To edit a template, select the template you want to edit from the list of templates. Next, select the **Actions** dropdown list and select the **Edit** option.

### Deleting a template

To delete a template, select the template you want to delete from the list of templates. Next, select the **Actions** dropdown list and select the **Delete** option.

## Aliases

An alias is a virtual index name that can point to one or more indexes. If your data is spread across multiple indexes, rather than keeping track of which indexes to query, you can create an alias and query it instead.
Expand Down Expand Up @@ -193,7 +210,7 @@ You can also disable and enable rollup jobs by selecting the corresponding butto

## Transform jobs

Using the admin UI in OpenSearch Dashboards, you can create, start, stop, and complete operations with [transform]({{site.url}}{{site.baseurl}}/im-plugin/index-transforms/transforms-apis/) jobs.
You can create, start, stop, and complete operations with [transform]({{site.url}}{{site.baseurl}}/im-plugin/index-transforms/transforms-apis/) jobs.

To create a transform job, perform the following steps:

Expand Down Expand Up @@ -223,3 +240,4 @@ You can check the status of the reindex, shrink, and split operations because th
## Error handling

Similar to API calls, if the operation fails immediately, you will be notified with an error message. However, if it is a long-running operation, you will be notified of the failure at the time of failure, or you can check the index status on the **Indices** page.

10 changes: 5 additions & 5 deletions _dashboards/admin-ui-index/index.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
---
layout: default
title: Admin UI for index operations in OpenSearch Dashboards
nav_order: 1
title: Index and snapshot management in OpenSearch Dashboards
nav_order: 15
has_children: true
redirect_from:
- /dashboards/admin-ui-index/
---

# Admin UI for index operations in OpenSearch Dashboards
# Index and snapshot management in OpenSearch Dashboards
Introduced 2.5
{: .label .label-purple }

The admin user interface (UI) for index operation allows you to perform common index operations through a user-friendly interface and prevent unintentional changes.
Previously, users relied on REST APIs or YAML configurations for basic administrative operations and interventions. This release takes the first step toward a unified administration panel in OpenSearch Dashboards with the launch of several index management UI enhancements. The new interface provides a more user-friendly way to run common indexing and data stream operations. Now you can perform create, read, update, and delete (CRUD) and mapping operations for indexes, index templates, and aliases through the UI. Additionally, you can open, close, reindex, shrink, and split indexes. The UI runs index status and data validation before submitting requests and lets you compare changes with previously saved settings before making updates.

The admin UI interface allows you to perform basic cluster administrative operations, or to view cluster health metrics and status. While there have been some administration user experiences in OpenSearch Dashboards (ISM, Alerting on HTTP inputs, Security, etc.), there is now a unified web UI experience for cluster administrators to administer their deployments.
<img src="{{site.url}}{{site.baseurl}}/images/admin-ui-index/admin-UI-preview.gif" alt="Index management demo gif">{: .img-fluid}
15 changes: 8 additions & 7 deletions _dashboards/admin-ui-index/sm-dashboards.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
layout: default
title: Using Snapshot Management in OpenSearch Dashboards
parent: Admin UI for index operations in OpenSearch Dashboards
nav_order: 30
title: Snapshot management
parent: Index and snapshot management in OpenSearch Dashboards
nav_order: 17
---

# Using Snapshot Management in OpenSearch Dashboards
# Snapshot management

You can set up Snapshot Management (SM) in OpenSearch Dashboards.

Expand All @@ -15,13 +15,13 @@ Snapshots have two main uses:

1. Recovering from failure

For example, if cluster health goes red, you might restore the red indexes from a snapshot.
For example, if cluster health goes red, you might restore the red indexes from a snapshot.

2. Migrating from one cluster to another

For example, if you’re moving from a proof-of-concept to a production cluster, you might take a snapshot of the former and restore it on the latter.
For example, if you’re moving from a proof-of-concept to a production cluster, you might take a snapshot of the former and restore it on the latter.

You can take and restore snapshots using the admin UI in Dashboards.
You can take and restore snapshots using snapshot management in OpenSearch Dashboards.

If you need to automate taking snapshots, you can use a snapshot policy.

Expand Down Expand Up @@ -173,3 +173,4 @@ The **Delete** button [deletes]({{site.url}}{{site.baseurl}}/api-reference/snaps
After the restore operation is complete, the restored indexes are listed in the **Indices** panel. To view the indexes, in the left panel, under **Index Management**, select **Indices**.

<img src="{{site.url}}{{site.baseurl}}/images/restore-snapshot/restore-snapshot-indices-panel.png" alt="View Indices">{: .img-fluid}

8 changes: 5 additions & 3 deletions _dashboards/maps.md → _dashboards/maps/maps.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
---
layout: default
title: Maps
nav_order: 15
title: Creating maps in OpenSearch Dashboards
nav_order: 70
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
redirect_from:
- /dashboards/maps/
---

# Maps
# Creating maps in OpenSearch Dashboards

With OpenSearch Dashboards, you can create maps to visualize your geographical data. OpenSearch lets you construct map visualizations with multiple layers, combining data across different indexes. You can build each layer from a different index pattern. Additionally, you can configure maps to show specific data at different zoom levels. OpenSearch maps are powered by the OpenSearch maps service, which uses vector tiles to render maps.

Expand Down
39 changes: 39 additions & 0 deletions _dashboards/maps/maptiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
layout: default
title: Configuring the Web Map Service
parent: Creating maps in OpenSearch Dashboards
nav_order: 10
redirect_from:
- /docs/opensearch-dashboards/maptiles/
---

{%- comment -%}The `/docs/opensearch-dashboards/maptiles/` redirect is specifically to support the UI links in OpenSearch Dashboards 1.0.0.{%- endcomment -%}

# Configuring the Web Map Service

OpenSearch Dashboards includes default map tiles. If you need specialized maps, you can use the [Web Map Service (WMS)](https://www.ogc.org/standards/wms).

Some map services have licensing fees or restrictions. You're responsible for all such considerations on any map server that you specify.
{: .note }

## Try it: Configuring your Dashboards and WMS

Follow these steps to configure your Dashboards and WMS.

1. Open OpenSearch Dashboards at `https://<host>:<port>`. For example, [https://localhost:5601](https://localhost:5601).
2. If necessary, log in.
3. Choose **Management** and **Advanced Settings**.
4. Locate `visualization:tileMap:WMSdefaults`.
5. Change `enabled` to true, and add the URL of a valid WMS map server:

```json
{
"enabled": true,
"url": "<wms-map-server-url>",
"options": {
"format": "image/png",
"transparent": true
}
}
```
Copy link
Member

@junqiu-lei junqiu-lei Jan 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the new maps plugin, we add WMS or TMS(tile map service) map by adding a custom layer in map page, instead of using Advanced Settings.


Loading