diff --git a/open-api.yaml b/open-api.yaml index 0acff98c..ca9a4ac4 100644 --- a/open-api.yaml +++ b/open-api.yaml @@ -2,7 +2,7 @@ openapi: 3.1.0 info: title: Meilisearch Core API description: 'Search documents, configure and manage the Meilisearch engine.' - version: 1.4.0 + version: 1.5.0 contact: name: Meilisearch email: bonjour@Meilisearch.com @@ -1097,6 +1097,7 @@ components: - stats.get - metrics.get - dumps.create + - snapshots.create - version - keys.get - keys.create @@ -1558,6 +1559,14 @@ components: status: enqueued type: dumpCreation enqueuedAt: '2021-01-01T09:39:00.000000Z' + 202_snapshotCreation: + summary: 202 Accepted snapshotCreation response example + value: + taskUid: 0 + indexUid: null + status: enqueued + type: snapshotCreation + enqueuedAt: '2021-01-01T09:39:00.000000Z' 202_taskCancelation: summary: 202 Accepted taskCancelation response example value: @@ -1631,6 +1640,13 @@ tags: During a [dump export](https://docs.meilisearch.com/reference/api/dump.html#create-a-dump), all indexes of the current instance are exported—together with their documents and settings—and saved as a single `.dump` file. During a dump import, all indexes contained in the indicated `.dump` file are imported along with their associated documents and settings. Any existing index with the same uid as an index in the dump file will be overwritten. Dump imports are [performed at launch](https://docs.meilisearch.com/reference/features/configuration.html#import-dump) using an option. + - name: Snapshots + description: | + The `snapshots` route allows the creation of database snapshots. Snapshots are `.snapshot` files that can be used to launch Meilisearch. + Creating a snapshot is also referred to as exporting it, whereas launching Meilisearch with a snapshot is referred to as importing it. + During a snapshot export, all indexes of the current instance are exported—together with their documents and settings—and saved as a single `.snapshot` file. + During a snapshot import, all indexes contained in the indicated `.snapshot` file are imported along with their associated documents and settings. + Snapshot imports are performed at launch using an option. - name: Experimental description: | The `experimental-features` endpoint allows enabling and disabling [experimental features](https://github.com/meilisearch/engine-team/blob/main/resources/experimental-features.md) at runtime. @@ -1658,6 +1674,28 @@ paths: $ref: '#/components/examples/202_dumpCreation' '401': $ref: '#/components/responses/401' + /snapshots: + post: + operationId: snapshots.create + summary: Create a Snapshot + description: | + Triggers a snapshot creation process. Once the process is complete, a snapshot is created in the snapshot directory. If the snapshot directory does not exist yet, it will be created. + tags: + - Snapshots + security: + - apiKey: [] + responses: + '202': + description: Accepted + content: + application/json: + schema: + $ref: '#/components/responses/202' + examples: + '202': + $ref: '#/components/examples/202_snapshotCreation' + '401': + $ref: '#/components/responses/401' /health: get: operationId: health.check @@ -2297,7 +2335,7 @@ paths: parameters: - $ref: '#/components/parameters/fields' delete: - operationId: indexes.documents.remove + operationId: indexes.documents.removeOne summary: Delete one document description: Delete one document based on its unique id. tags: diff --git a/text/0034-telemetry-policies.md b/text/0034-telemetry-policies.md index 9d3983f1..4431d954 100644 --- a/text/0034-telemetry-policies.md +++ b/text/0034-telemetry-policies.md @@ -65,6 +65,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | NonSeparatorTokens Updated | Occurs when non separator tokens are updated via `PUT` — `/indexes/:indexUid/settings/non-separator-tokens`. | | Synonyms Updated | Occurs when synonyms are updated via `PUT` — `/indexes/:indexUid/settings/synonyms`. | | Dump Created | Occurs when a dump is created via `POST` - `/dumps`. | +| Snapshot Created | Occurs when a snapshot is created via `POST` - `/snapshots`. | | Tasks Canceled | Occurs when tasks are requested to be canceled via `POST` - `/tasks/cancel`. | | Tasks Deleted | Occurs when tasks are requested to be deleted via `DELETE`- `/tasks`. | | Tasks Seen | Aggregated event on all received requests via the `GET` - `/tasks` route during one hour or until a batch size reaches `500Kb`. | @@ -117,7 +118,7 @@ The collected data is sent to [Segment](https://segment.com/). Segment is a plat | `stats.database_size` | Database size. Expressed in `Bytes` | 2621440 | Every hour | | `stats.indexes_number` | Number of indexes | 2 | Every hour | | `start_since_days` | Number of days since instance was launched | 365 | Every hour | -| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST`, `Experimental features Seen`, `Experimental features Updated` | +| `user_agent` | User-agent header encountered during one or more API calls | ["Meilisearch Ruby (v2.1)", "Ruby (3.0)"] | `Documents Searched POST`, `Documents Searched GET`, `Index Created`, `Index Updated`, `Documents Added`, `Documents Updated`, `Documents Deleted`, `Settings Updated`, `Ranking Rules Updated`, `SortableAttributes Updated`, `FilterableAttributes Updated`, `SearchableAttributes Updated`, `TypoTolerance Updated`, `Pagination Updated`, `Faceting Updated`, `DistinctAttribute Updated`, `DisplayedAttributes Updated`, `StopWords Updated`, `Synonyms Updated`, `Dump Created`, `Snapshot Created`, `Tasks Seen`, `Stats Seen`, `Health Seen`, `Version Seen`, `Documents Searched by Multi-Search POST`, `Experimental features Seen`, `Experimental features Updated` | | `requests.99th_response_time` | Highest latency from among the fastest 99% of successful search requests | 57ms | `Documents Searched POST`, `Documents Searched GET`| | `requests.total_succeeded` | Total number of successful requests in this batch | 3456 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | | `requests.total_failed` | Total number of failed requests in this batch | 24 | `Documents Searched POST`, `Documents Searched GET`, `Documents Searched by Multi-Search POST` | @@ -629,6 +630,12 @@ This property allows us to gather essential information to better understand on |---------------|-------------|---------| | user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | +## `Snapshot Created` + +| Property name | Description | Example | +|---------------|-------------|---------| +| user_agent | Represents the user-agent encountered on this call. | `["Meilisearch Ruby (v2.1)", "Ruby (3.0)"]` | + ## `Tasks Seen` > The Tasks Seen event is sent once an hour or when a batch reaches the maximum size of `500Kb`. diff --git a/text/0085-api-keys.md b/text/0085-api-keys.md index 685a7611..f54f2e71 100644 --- a/text/0085-api-keys.md +++ b/text/0085-api-keys.md @@ -330,6 +330,7 @@ Create an API key. | stats.get | Provides access to `GET` `/stats`. **⚠️Non-authorized `indexes` are omitted from the response on `/stats`**. Also add access to `GET` `/indexes/:authorizedIndexes/stats`. | | metrics.get | Provides access to `GET` `/metrics` route. **A restriction on `indexes` stops you from calling the route.** | | dumps.create | Provides access to `POST` `/dumps` route. **As dumps are not scoped by indexes, a restriction on `indexes` does not affect this action.** | +| snapshots.create | Provides access to `POST` `/snapshots` route. **As snapshots are not scoped by indexes, a restriction on `indexes` does not affect this action.** | | version | Provides access to `GET` `/version` route. | | keys.get | Provides access to `GET` `/keys` route. | | keys.create | Provides access to `POST` `/keys` route. | diff --git a/text/0246-facet-search-api.md b/text/0246-facet-search-api.md index e4418195..310fa73a 100644 --- a/text/0246-facet-search-api.md +++ b/text/0246-facet-search-api.md @@ -45,8 +45,8 @@ If a master key is used to secure a Meilisearch instance, the auth layer returns `facetName` contains the facet name to search values on. -- 🔴 Omitting `facetName` returns a `missing_facet_search_facet_name`(0061-error-format-and-definitions.md#missing_facet_search_facet_name) error. -- 🔴 Sending a value with a different type than `String` for `facetName` returns a [missing_facet_search_facet_name](0061-error-format-and-definitions.md#minvalid_facet_search_facet_name) error. +- 🔴 Omitting `facetName` returns a [missing_facet_search_facet_name](0061-error-format-and-definitions.md#missing_facet_search_facet_name) error. +- 🔴 Sending a value with a different type than `String` for `facetName` returns a [invalid_facet_search_facet_name](0061-error-format-and-definitions.md#invalid_facet_search_facet_name) error. - 🔴 Sending a field not defined as a `filterableAttributes` for `facetname` returns an [invalid_facet_search_facet_name](0061-error-format-and-definitions.md#invalid_facet_search_facet_name) error. #### 3.1.2. `facetQuery` diff --git a/text/0258-snapshots-api.md b/text/0258-snapshots-api.md new file mode 100644 index 00000000..1840cc42 --- /dev/null +++ b/text/0258-snapshots-api.md @@ -0,0 +1,75 @@ +# Snapshots API and CLI + +## 1. Summary + +A snapshot is a compressed file containing an export of a MeiliSearch instance. It contains all indexes, documents, settings, tasks, and API keys. + +## 2. Motivation + +The snapshots exist to start a MeiliSearch instance from a database as fast as possible. It can be a helpful tool for loading a production state on a staging server to make changes and test them before propagating them to production. + +## 3. Functional Specification + +### 3.1. Summary Key Points + +- A snapshot creation can be scheduled from the MeiliSearch API using the `POST - /snapshots` endpoint. +- A snapshot creation status can be tracked using the `GET - /tasks/:task_uid` endpoint. +- MeiliSearch will autobatch all your snapshots requests into one. +- By default, snapshots are created in a folder named `snapshots`, and can be found in the same directory as the MeiliSearch binary. +- The `snapshots` directory can be customized using the `--snapshot-dir` configuration option. If the snapshot directory does not already exist when the snapshot creation process is called, MeiliSearch will create it. +- A `.snapshot` file can be imported using the `--import-snapshot` command-line flag. +- The MeiliSearch server starts when the snapshot is fully imported. +- By default, importing a snapshot when a database already exists (a non-empty `data.ms` folder in the same directory as the MeiliSearch binary) will stop the process and throw an error. +- When using the command-line flag `--ignore-snapshot-if-db-exists=true`, MeiliSearch will use the existing database to start an instance instead of throwing an error. The snapshot will be ignored. +- By default, trying to import a snapshot that does not exist, will stop the process and throw an error. +- When using the command-line flag `--ignore-missing-snapshot`, MeiliSearch will continue its process and not throw an error. +- When a snapshot is being imported, the http API is not available. Meilisearch can't receive read or write requests. +- `snapshotCreation` task takes priority over enqueued `tasks`. This means that if a `snapshotCreation` task is created, it will be directly processed when the current processing task finishes even if other tasks have been enqueued before. + +--- + +### 3.2. Snapshots API Definition + +#### 3.2.1. POST `/snapshot` + +Create a snapshot + +##### 3.2.1.1. Body Payload +N/A + +##### 3.2.1.2. Response Definition + +When the request is successful, Meilisearch returns the HTTP code `202 Accepted`. The response's content is the summarized representation of the received asynchronous task. + +See [Summarized `task` Object for `202 Accepted`](0060-tasks-api.md#summarized-task-object-for-202-accepted). + +The name of the generated snapshot is the database path (`data.ms` by default), joined by the `.snapshot` extension. By default, it's `data.ms.snapshot`. + +##### 3.2.1.3. Errors + +- 🔴 If Meilisearch is secured, accessing this route without the `Authorization` header returns a [missing_authorization_header](0061-error-format-and-definitions.md#missing_authorization_header) error. +- 🔴 If Meilisearch is secured, accessing this route with a key that does not have permissions (i.e. other than the master-key) returns an [invalid_api_key](0061-error-format-and-definitions.md#invalid_api_key) error. + +--- + +### 3.3. CLI Definition + +You can find the CLI informations about the snapshots [here](text/0119-instance-options.md) + +## 4. Technical Aspects + +### 4.1. Snapshot Creation + +When a snapshot is being created, the task queue can receive other future operations to perform. + +### 4.2. Importing a snapshot + +When a snapshot is being imported, the http API is not available. Meilisearch can't receive read or write requests. + +### 4.3. Snapshot Creation Task Priority + +Snapshot creation tasks have priority over other task types. If a `snapshotCreation` task is enqueued, it will be directly processed when the current processing task finishes even if other tasks have been enqueued before. + +## 5. Future Possibilities + +- Give information about who created the task (is it scheduled or created on a user demand) in the details.