From b56c1880dc632a02a6a97a1631192a2157802d37 Mon Sep 17 00:00:00 2001 From: Petros Angelatos Date: Fri, 14 Nov 2025 11:41:01 +0200 Subject: [PATCH 1/2] doc: document new source syntax Signed-off-by: Petros Angelatos --- .../postgres/create_source_cloud.yml | 38 ++++++++++++++++++- 1 file changed, 36 insertions(+), 2 deletions(-) diff --git a/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml b/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml index 11ee30372f654..72e45d0e0ac7e 100644 --- a/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml +++ b/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml @@ -5,17 +5,51 @@ instance and start ingesting data from the publication you created [earlier](#2-create-a-publication-and-a-replication-user): - code: | + {{< tabs >}} + {{< tab "Current Syntax">}} + + ```mzsql + -- Step 1: Create the source + CREATE SOURCE mz_source + IN CLUSTER ingest_postgres + FROM POSTGRES CONNECTION pg_connection (PUBLICATION 'mz_source'); + + -- Step 2: Create tables from the source + CREATE TABLE table1 FROM SOURCE mz_source (REFERENCE table1); + CREATE TABLE table2 FROM SOURCE mz_source (REFERENCE table2); + ``` + + {{< /tab >}} + {{< tab "Legacy Syntax">}} + + ```mzsql CREATE SOURCE mz_source IN CLUSTER ingest_postgres FROM POSTGRES CONNECTION pg_connection (PUBLICATION 'mz_source') FOR ALL TABLES; + ``` + + {{< warning >}} + This syntax is deprecated and may be removed in a future version. Please use the current syntax for new implementations. + {{< /warning >}} + + {{< /tab >}} + {{< /tabs >}} - name: "create-source-options" description: | + **Current Syntax:** + + - By default, the source will be created in the active cluster; to use a different cluster, use the `IN CLUSTER` clause. + - After creating the source, you can query `mz_internal.mz_source_references` to see available tables in the publication. + - Create individual tables using `CREATE TABLE FROM SOURCE (REFERENCE )`. + - To handle [unsupported data types](#supported-types), use `WITH (TEXT COLUMNS = [])` or `WITH (EXCLUDE COLUMNS = [])` in the `CREATE TABLE` statement. + + **Legacy Syntax:** + - By default, the source will be created in the active cluster; to use a different cluster, use the `IN CLUSTER` clause. - To ingest data from specific schemas or tables, use the `FOR SCHEMAS (,)` or `FOR TABLES (, )` options instead of `FOR ALL TABLES`. - - To handle [unsupported data types](#supported-types), use the `TEXT COLUMNS` or `EXCLUDE COLUMNS` options. + - To handle [unsupported data types](#supported-types), use the `TEXT COLUMNS` or `EXCLUDE COLUMNS` options in the `CREATE SOURCE` statement. - name: "schema-changes" description: | From 7f8cae56c625d7457a72be36f325425bd1957acd Mon Sep 17 00:00:00 2001 From: kay-kim Date: Fri, 14 Nov 2025 11:45:53 -0500 Subject: [PATCH 2/2] tweaks --- .../postgres/create_source_cloud.yml | 63 ++++++++++--------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml b/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml index 72e45d0e0ac7e..671f3b602db40 100644 --- a/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml +++ b/doc/user/data/examples/ingest_data/postgres/create_source_cloud.yml @@ -1,27 +1,16 @@ - name: "create-source" description: | - Once you have created the connection, you can use the connection in the - [`CREATE SOURCE`](/sql/create-source/) command to connect to your PostgreSQL - instance and start ingesting data from the publication you created - [earlier](#2-create-a-publication-and-a-replication-user): - {{< tabs >}} - {{< tab "Current Syntax">}} - ```mzsql - -- Step 1: Create the source - CREATE SOURCE mz_source - IN CLUSTER ingest_postgres - FROM POSTGRES CONNECTION pg_connection (PUBLICATION 'mz_source'); - - -- Step 2: Create tables from the source - CREATE TABLE table1 FROM SOURCE mz_source (REFERENCE table1); - CREATE TABLE table2 FROM SOURCE mz_source (REFERENCE table2); - ``` + {{< tabs >}} - {{< /tab >}} {{< tab "Legacy Syntax">}} + Once you have created the connection, you can use the connection in the + [`CREATE SOURCE`](/sql/create-source/postgres/) command to connect to your + PostgreSQL instance and start ingesting data from the publication you + created [earlier](#2-create-a-publication-and-a-replication-user): + ```mzsql CREATE SOURCE mz_source IN CLUSTER ingest_postgres @@ -29,27 +18,39 @@ FOR ALL TABLES; ``` - {{< warning >}} - This syntax is deprecated and may be removed in a future version. Please use the current syntax for new implementations. - {{< /warning >}} + - By default, the source will be created in the active cluster; to use a different cluster, use the `IN CLUSTER` clause. + - To ingest data from specific schemas or tables, use the `FOR SCHEMAS (,)` or `FOR TABLES (, )` options instead of `FOR ALL TABLES`. + - To handle [unsupported data types](#supported-types), use the `TEXT + COLUMNS` or `EXCLUDE COLUMNS` options in the `CREATE SOURCE` statement. {{< /tab >}} - {{< /tabs >}} + {{< tab "New Syntax">}} -- name: "create-source-options" - description: | - **Current Syntax:** + Once you have created the connection, you can: + - use the connection in the [`CREATE SOURCE`](/sql/create-source/postgres/) + command to connect to your PostgreSQL instance and + - use the source in the [`CREATE TABLE`](/sql/create-table/) to create the + tables in Materialize and start ingesting data from the publication you + created [earlier](#2-create-a-publication-and-a-replication-user): - - By default, the source will be created in the active cluster; to use a different cluster, use the `IN CLUSTER` clause. - - After creating the source, you can query `mz_internal.mz_source_references` to see available tables in the publication. - - Create individual tables using `CREATE TABLE FROM SOURCE (REFERENCE )`. - - To handle [unsupported data types](#supported-types), use `WITH (TEXT COLUMNS = [])` or `WITH (EXCLUDE COLUMNS = [])` in the `CREATE TABLE` statement. + ```mzsql + -- Step 1: Create the source + CREATE SOURCE mz_source + IN CLUSTER ingest_postgres + FROM POSTGRES CONNECTION pg_connection (PUBLICATION 'mz_source'); - **Legacy Syntax:** + -- Step 2: Create tables from the source + CREATE TABLE table1 FROM SOURCE mz_source (REFERENCE table1); + CREATE TABLE table2 FROM SOURCE mz_source (REFERENCE table2); + ``` - By default, the source will be created in the active cluster; to use a different cluster, use the `IN CLUSTER` clause. - - To ingest data from specific schemas or tables, use the `FOR SCHEMAS (,)` or `FOR TABLES (, )` options instead of `FOR ALL TABLES`. - - To handle [unsupported data types](#supported-types), use the `TEXT COLUMNS` or `EXCLUDE COLUMNS` options in the `CREATE SOURCE` statement. + - After creating the source, you can query `mz_internal.mz_source_references` to see available tables in the publication. + - Create individual tables using [`CREATE TABLE FROM SOURCE (REFERENCE )`](/sql/create-table/). + - To handle [unsupported data types](#supported-types), use `WITH (TEXT COLUMNS = [])` or `WITH (EXCLUDE COLUMNS = [])` in the [`CREATE TABLE`](/sql/create-table/) statement. + + {{< /tab >}} + {{< /tabs >}} - name: "schema-changes" description: |