Skip to content
This repository has been archived by the owner on Mar 21, 2024. It is now read-only.

Release v1.5.0 #259

Merged
merged 3 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 40 additions & 2 deletions open-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ openapi: 3.1.0
info:
title: Meilisearch Core API
description: 'Search documents, configure and manage the Meilisearch engine.'
version: 1.4.0
version: 1.5.0
contact:
name: Meilisearch
email: bonjour@Meilisearch.com
Expand Down Expand Up @@ -1097,6 +1097,7 @@ components:
- stats.get
- metrics.get
- dumps.create
- snapshots.create
- version
- keys.get
- keys.create
Expand Down Expand Up @@ -1558,6 +1559,14 @@ components:
status: enqueued
type: dumpCreation
enqueuedAt: '2021-01-01T09:39:00.000000Z'
202_snapshotCreation:
summary: 202 Accepted snapshotCreation response example
value:
taskUid: 0
indexUid: null
status: enqueued
type: snapshotCreation
enqueuedAt: '2021-01-01T09:39:00.000000Z'
202_taskCancelation:
summary: 202 Accepted taskCancelation response example
value:
Expand Down Expand Up @@ -1631,6 +1640,13 @@ tags:
During a [dump export](https://docs.meilisearch.com/reference/api/dump.html#create-a-dump), all indexes of the current instance are exported—together with their documents and settings—and saved as a single `.dump` file.
During a dump import, all indexes contained in the indicated `.dump` file are imported along with their associated documents and settings. Any existing index with the same uid as an index in the dump file will be overwritten.
Dump imports are [performed at launch](https://docs.meilisearch.com/reference/features/configuration.html#import-dump) using an option.
- name: Snapshots
description: |
The `snapshots` route allows the creation of database snapshots. Snapshots are `.snapshot` files that can be used to launch Meilisearch.
Creating a snapshot is also referred to as exporting it, whereas launching Meilisearch with a snapshot is referred to as importing it.
During a snapshot export, all indexes of the current instance are exported—together with their documents and settings—and saved as a single `.snapshot` file.
During a snapshot import, all indexes contained in the indicated `.snapshot` file are imported along with their associated documents and settings.
Snapshot imports are performed at launch using an option.
- name: Experimental
description: |
The `experimental-features` endpoint allows enabling and disabling [experimental features](https://github.com/meilisearch/engine-team/blob/main/resources/experimental-features.md) at runtime.
Expand Down Expand Up @@ -1658,6 +1674,28 @@ paths:
$ref: '#/components/examples/202_dumpCreation'
'401':
$ref: '#/components/responses/401'
/snapshots:
post:
operationId: snapshots.create
summary: Create a Snapshot
description: |
Triggers a snapshot creation process. Once the process is complete, a snapshot is created in the snapshot directory. If the snapshot directory does not exist yet, it will be created.
tags:
- Snapshots
security:
- apiKey: []
responses:
'202':
description: Accepted
content:
application/json:
schema:
$ref: '#/components/responses/202'
examples:
'202':
$ref: '#/components/examples/202_snapshotCreation'
'401':
$ref: '#/components/responses/401'
/health:
get:
operationId: health.check
Expand Down Expand Up @@ -2297,7 +2335,7 @@ paths:
parameters:
- $ref: '#/components/parameters/fields'
delete:
operationId: indexes.documents.remove
operationId: indexes.documents.removeOne
summary: Delete one document
description: Delete one document based on its unique id.
tags:
Expand Down
9 changes: 8 additions & 1 deletion text/0034-telemetry-policies.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
| NonSeparatorTokens Updated | Occurs when non separator tokens are updated via `PUT` — `/indexes/:indexUid/settings/non-separator-tokens`. |
| Synonyms Updated | Occurs when synonyms are updated via `PUT` — `/indexes/:indexUid/settings/synonyms`. |
| Dump Created | Occurs when a dump is created via `POST` - `/dumps`. |
| Snapshot Created | Occurs when a snapshot is created via `POST` - `/snapshots`. |
| Tasks Canceled | Occurs when tasks are requested to be canceled via `POST` - `/tasks/cancel`. |
| Tasks Deleted | Occurs when tasks are requested to be deleted via `DELETE`- `/tasks`. |
| Tasks Seen | Aggregated event on all received requests via the `GET` - `/tasks` route during one hour or until a batch size reaches `500Kb`. |
Expand Down Expand Up @@ -117,7 +118,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat
| `stats.database_size` | Database size. Expressed in `Bytes` | 2621440 | Every hour |
| `stats.indexes_number` | Number of indexes | 2 | Every hour |
| `start_since_days` | Number of days since instance was launched | 365 | Every hour |
| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST`, `Experimental features Seen`, `Experimental features Updated` |
| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Snapshot Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST`, `Experimental features Seen`, `Experimental features Updated` |
| `requests.99th_response_time` | Highest latency from among the fastest 99% of successful search requests | 57ms | `Documents Searched POST`, `Documents Searched GET`|
| `requests.total_succeeded` | Total number of successful requests in this batch | 3456 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` |
| `requests.total_failed` | Total number of failed requests in this batch | 24 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` |
Expand Down Expand Up @@ -629,6 +630,12 @@ This property allows us to gather essential information to better understand on
|---------------|-------------|---------|
| user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` |

## `Snapshot Created`

| Property name | Description | Example |
|---------------|-------------|---------|
| user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` |

## `Tasks Seen`

> The Tasks Seen event is sent once an hour or when a batch reaches the maximum size of `500Kb`.
Expand Down
1 change: 1 addition & 0 deletions text/0085-api-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,7 @@ Create an API key.
| stats.get | Provides access to `GET` `/stats`. **⚠️Non-authorized `indexes` are omitted from the response on `/stats`**. Also add access to `GET` `/indexes/:authorizedIndexes/stats`. |
| metrics.get | Provides access to `GET` `/metrics` route. **A restriction on `indexes` stops you from calling the route.** |
| dumps.create | Provides access to `POST` `/dumps` route. **As dumps are not scoped by indexes, a restriction on `indexes` does not affect this action.** |
| snapshots.create | Provides access to `POST` `/snapshots` route. **As snapshots are not scoped by indexes, a restriction on `indexes` does not affect this action.** |
| version | Provides access to `GET` `/version` route. |
| keys.get | Provides access to `GET` `/keys` route. |
| keys.create | Provides access to `POST` `/keys` route. |
Expand Down
4 changes: 2 additions & 2 deletions text/0246-facet-search-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ If a master key is used to secure a Meilisearch instance, the auth layer returns

`facetName` contains the facet name to search values on.

- 🔴 Omitting `facetName` returns a `missing_facet_search_facet_name`(0061-error-format-and-definitions.md#missing_facet_search_facet_name) error.
- 🔴 Sending a value with a different type than `String` for `facetName` returns a [missing_facet_search_facet_name](0061-error-format-and-definitions.md#minvalid_facet_search_facet_name) error.
- 🔴 Omitting `facetName` returns a [missing_facet_search_facet_name](0061-error-format-and-definitions.md#missing_facet_search_facet_name) error.
- 🔴 Sending a value with a different type than `String` for `facetName` returns a [invalid_facet_search_facet_name](0061-error-format-and-definitions.md#invalid_facet_search_facet_name) error.
- 🔴 Sending a field not defined as a `filterableAttributes` for `facetname` returns an [invalid_facet_search_facet_name](0061-error-format-and-definitions.md#invalid_facet_search_facet_name) error.

#### 3.1.2. `facetQuery`
Expand Down
75 changes: 75 additions & 0 deletions text/0258-snapshots-api.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Snapshots API and CLI

## 1. Summary

A snapshot is a compressed file containing an export of a MeiliSearch instance. It contains all indexes, documents, settings, tasks, and API keys.

## 2. Motivation

The snapshots exist to start a MeiliSearch instance from a database as fast as possible. It can be a helpful tool for loading a production state on a staging server to make changes and test them before propagating them to production.

## 3. Functional Specification

### 3.1. Summary Key Points

- A snapshot creation can be scheduled from the MeiliSearch API using the `POST - /snapshots` endpoint.
- A snapshot creation status can be tracked using the `GET - /tasks/:task_uid` endpoint.
- MeiliSearch will autobatch all your snapshots requests into one.
- By default, snapshots are created in a folder named `snapshots`, and can be found in the same directory as the MeiliSearch binary.
- The `snapshots` directory can be customized using the `--snapshot-dir` configuration option. If the snapshot directory does not already exist when the snapshot creation process is called, MeiliSearch will create it.
- A `.snapshot` file can be imported using the `--import-snapshot` command-line flag.
- The MeiliSearch server starts when the snapshot is fully imported.
- By default, importing a snapshot when a database already exists (a non-empty `data.ms` folder in the same directory as the MeiliSearch binary) will stop the process and throw an error.
- When using the command-line flag `--ignore-snapshot-if-db-exists=true`, MeiliSearch will use the existing database to start an instance instead of throwing an error. The snapshot will be ignored.
- By default, trying to import a snapshot that does not exist, will stop the process and throw an error.
- When using the command-line flag `--ignore-missing-snapshot`, MeiliSearch will continue its process and not throw an error.
- When a snapshot is being imported, the http API is not available. Meilisearch can't receive read or write requests.
- `snapshotCreation` task takes priority over enqueued `tasks`. This means that if a `snapshotCreation` task is created, it will be directly processed when the current processing task finishes even if other tasks have been enqueued before.

---

### 3.2. Snapshots API Definition

#### 3.2.1. POST `/snapshot`

Create a snapshot

##### 3.2.1.1. Body Payload
N/A

##### 3.2.1.2. Response Definition

When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task.

See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted).

The name of the generated snapshot is the database path (`data.ms` by default), joined by the `.snapshot` extension. By default, it's `data.ms.snapshot`.

##### 3.2.1.3. Errors

- 🔴 If Meilisearch is secured, accessing this route without the `Authorization` header returns a [missing_authorization_header](0061-error-format-and-definitions.md#missing_authorization_header) error.
- 🔴 If Meilisearch is secured, accessing this route with a key that does not have permissions (i.e. other than the master-key) returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error.

---

### 3.3. CLI Definition

You can find the CLI informations about the snapshots [here](text/0119-instance-options.md)

## 4. Technical Aspects

### 4.1. Snapshot Creation

When a snapshot is being created, the task queue can receive other future operations to perform.

### 4.2. Importing a snapshot

When a snapshot is being imported, the http API is not available. Meilisearch can't receive read or write requests.

### 4.3. Snapshot Creation Task Priority

Snapshot creation tasks have priority over other task types. If a `snapshotCreation` task is enqueued, it will be directly processed when the current processing task finishes even if other tasks have been enqueued before.

## 5. Future Possibilities

- Give information about who created the task (is it scheduled or created on a user demand) in the details.
Loading