From 793fe66d2c0f99cfb09ed23e75a369a87e4a17ff Mon Sep 17 00:00:00 2001 From: Jimmy Ma Date: Tue, 26 Mar 2024 15:53:11 -0700 Subject: [PATCH 1/3] Update cloud record size limit doc --- .../managing-airbyte-cloud/understand-airbyte-cloud-limits.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md index f9e4b5467a84..9a006738390a 100644 --- a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md +++ b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md @@ -11,6 +11,8 @@ Understanding the following limitations will help you more effectively manage Ai * Max number of destinations in a workspace: 20* * Max number of streams that can be returned by a source in a discover call: 1K * Max number of streams that can be configured to sync in a single connection: 1K -* Size of a single record: 20MB +* Size of a single record: 20MB\** *Limits on workspaces, sources, and destinations do not apply to customers of [Powered by Airbyte](https://airbyte.com/solutions/powered-by-airbyte). To learn more [contact us](https://airbyte.com/talk-to-sales)! + +\**The effective maximum size of the record may vary based per destination. For example, maximum size of a record in MongoDB is 16MB, the destination connector will be responsible for ensuring consistent behavior. From 22ffb04a4d1c0e8b7b841868557298a8f9bb414a Mon Sep 17 00:00:00 2001 From: Jimmy Ma Date: Tue, 26 Mar 2024 16:18:24 -0700 Subject: [PATCH 2/3] Update docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md Co-authored-by: Evan Tahler --- .../managing-airbyte-cloud/understand-airbyte-cloud-limits.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md index 9a006738390a..df13d5d453fa 100644 --- a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md +++ b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md @@ -15,4 +15,6 @@ Understanding the following limitations will help you more effectively manage Ai *Limits on workspaces, sources, and destinations do not apply to customers of [Powered by Airbyte](https://airbyte.com/solutions/powered-by-airbyte). To learn more [contact us](https://airbyte.com/talk-to-sales)! -\**The effective maximum size of the record may vary based per destination. For example, maximum size of a record in MongoDB is 16MB, the destination connector will be responsible for ensuring consistent behavior. +\**The effective maximum size of the record may vary based per destination. Some destinations may fail to sync if a record cannot be stored, but Destinations which support [typing and deduping](/core-concepts/typing-deduping) will adjust your record so that the sync does not fail, given the database/file constraints. For example, the maximum size of a record in MongoDB is 16MB - records larger than that will need to be modified. + +At the very least, primary keys and cursors will be maintained. Any modifications to the record will be stored within `airbyte_meta.changes` for your review within the destination. From 59105a64f75ef2dbab243c7db8f0a3ac082f82e1 Mon Sep 17 00:00:00 2001 From: evantahler Date: Tue, 26 Mar 2024 16:59:51 -0700 Subject: [PATCH 3/3] fix link and formatting --- .../understand-airbyte-cloud-limits.md | 26 ++++++++++++------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md index df13d5d453fa..86d9ef7188e2 100644 --- a/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md +++ b/docs/cloud/managing-airbyte-cloud/understand-airbyte-cloud-limits.md @@ -6,15 +6,23 @@ products: cloud Understanding the following limitations will help you more effectively manage Airbyte Cloud. -* Max number of workspaces per user: 3* -* Max number of instances of the same source connector: 10* -* Max number of destinations in a workspace: 20* -* Max number of streams that can be returned by a source in a discover call: 1K -* Max number of streams that can be configured to sync in a single connection: 1K -* Size of a single record: 20MB\** +- Max number of workspaces per user: 3\* +- Max number of instances of the same source connector: 10\* +- Max number of destinations in a workspace: 20\* +- Max number of streams that can be returned by a source in a discover call: 1K +- Max number of streams that can be configured to sync in a single connection: 1K +- Size of a single record: 20MB\*\* -*Limits on workspaces, sources, and destinations do not apply to customers of [Powered by Airbyte](https://airbyte.com/solutions/powered-by-airbyte). To learn more [contact us](https://airbyte.com/talk-to-sales)! +--- -\**The effective maximum size of the record may vary based per destination. Some destinations may fail to sync if a record cannot be stored, but Destinations which support [typing and deduping](/core-concepts/typing-deduping) will adjust your record so that the sync does not fail, given the database/file constraints. For example, the maximum size of a record in MongoDB is 16MB - records larger than that will need to be modified. +\* Limits on workspaces, sources, and destinations do not apply to customers of +[Powered by Airbyte](https://airbyte.com/solutions/powered-by-airbyte). To learn more +[contact us](https://airbyte.com/talk-to-sales)! -At the very least, primary keys and cursors will be maintained. Any modifications to the record will be stored within `airbyte_meta.changes` for your review within the destination. +\*\* The effective maximum size of the record may vary based per destination. Some destinations may +fail to sync if a record cannot be stored, but Destinations which support +[typing and deduping](/using-airbyte/core-concepts/typing-deduping) will adjust your record so that +the sync does not fail, given the database/file constraints. For example, the maximum size of a +record in MongoDB is 16MB - records larger than that will need to be modified. At the very least, +primary keys and cursors will be maintained. Any modifications to the record will be stored within +`airbyte_meta.changes` for your review within the destination.