Fix #12677: Added Synapse Connector - docs and side docs (#17041)

ingestion/src/metadata/examples/workflows/synapse.yaml

@@ -0,0 +1,24 @@
+source:
+  type: synapse
+  serviceName: local_synapse
+  serviceConnection:
+    config:
+      type: Synapse
+      database: master
+      username: username
+      password: password
+      hostPort: host.at.azuresynapse.net
+      driver: ODBC Driver 17 for SQL Server
+  sourceConfig:
+    config:
+      type: DatabaseMetadata
+sink:
+  type: metadata-rest
+  config: {}
+workflowConfig:
+  loggerLevel: INFO # DEBUG, INFO, WARN or ERROR
+  openMetadataServerConfig:
+    hostPort: http://localhost:8585/api
+    authProvider: openmetadata
+    securityConfig:
+      jwtToken: "eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImlzQm90IjpmYWxzZSwiaXNzIjoib3Blbi1tZXRhZGF0YS5vcmciLCJpYXQiOjE2NjM5Mzg0NjIsImVtYWlsIjoiYWRtaW5Ab3Blbm1ldGFkYXRhLm9yZyJ9.tS8um_5DKu7HgzGBzS1VTA5uUjKWOCU0B_j08WXBiEC0mr0zNREkqVfwFDD-d24HlNEbrqioLsBuFRiwIWKc1m_ZlVQbG7P36RUxhuv2vbSp80FKyNM-Tj93FDzq91jsyNmsQhyNv_fNr3TXfzzSPjHt8Go0FMMP66weoKMgW2PbXlhVKwEuXUHyakLLzewm9UMeQaEiRzhiTMU3UkLXcKbYEJJvfNFcLwSl9W8JCO_l0Yj3ud-qt_nQYEZwqW6u5nfdQllN133iikV4fM5QZsMCnm8Rq1mvLR0y9bmJiD7fwM1tmJ791TUWqmKaTnP49U493VanKpUAfzIiOiIbhg"      

ingestion/src/metadata/ingestion/connections/test_connections.py

@@ -277,7 +277,9 @@ def test_connection_engine_step(connection: Engine) -> None:
     Generic step to validate the connection against a db
     """
     with connection.connect() as conn:
-        conn.execute(ConnTestFn())
+        result = conn.execute(ConnTestFn())
+        if result:
+            result.fetchone()
 
 
 def test_connection_db_common(

openmetadata-docs/content/v1.5.x-SNAPSHOT/connectors/database/synapse/index.md

@@ -0,0 +1,100 @@
+---
+title: Synapse
+slug: /connectors/database/synapse
+---
+
+{% connectorDetailsHeader
+name="Synapse"
+stage="PROD"
+platform="Collate"
+availableFeatures=["Metadata", "Query Usage", "Lineage", "Column-level Lineage", "Data Profiler", "Data Quality", "Stored Procedures", "dbt"]
+unavailableFeatures=["Owners", "Tags"]
+/ %}
+
+In this section, we provide guides and references to use the Synapse connector.
+
+Configure and schedule Synapse metadata and profiler workflows from the OpenMetadata UI:
+
+- [Requirements](#requirements)
+- [Metadata Ingestion](#metadata-ingestion)
+    - [Service Name](#service-name)
+    - [Connection Options](#connection-options)
+    - [Metadata Ingestion Options](#metadata-ingestion-options)
+- [Troubleshooting](#troubleshooting)
+  - [Workflow Deployment Error](#workflow-deployment-error)
+- [Related](#related)
+
+{% partial file="/v1.5/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/database/synapse/yaml"} /%}
+
+## Requirements
+
+Make sure if you have whitelisted ingestion container IP on Azure SQL firewall rules. Checkout [this](https://learn.microsoft.com/en-us/azure/azure-sql/database/firewall-configure?view=azuresql#use-the-azure-portal-to-manage-server-level-ip-firewall-rules) document on how to whitelist your IP using azure portal.
+
+Synapse database user must grant `SELECT` privilege to fetch the metadata of tables and views.
+
+```sql
+-- Create a new user
+-- More details https://learn.microsoft.com/en-us/sql/t-sql/statements/create-user-transact-sql?view=sql-server-ver16
+CREATE USER Mary WITH PASSWORD = '********';
+-- Grant SELECT on table
+GRANT SELECT TO Mary;
+```
+
+## Metadata Ingestion
+
+{% partial 
+  file="/v1.5/connectors/metadata-ingestion-ui.md" 
+  variables={
+    connector: "Synapse", 
+    selectServicePath: "/images/v1.5/connectors/synapse/select-service.webp",
+    addNewServicePath: "/images/v1.5/connectors/synapse/add-new-service.webp",
+    serviceConnectionPath: "/images/v1.5/connectors/synapse/service-connection.webp",
+} 
+/%}
+
+{% stepsContainer %}
+{% extraContent parentTagName="stepsContainer" %}
+
+#### Connection Options
+
+- **Username**: Specify the User to connect to Synapse. It should have enough privileges to read all the metadata.
+- **Password**: Password to connect to Synapse.
+- **Host and Port**: Enter the fully qualified hostname and port number for your Synapse deployment in the Host and Port field.
+- **Database**: The database of the data source is an optional parameter, if you would like to restrict the metadata reading to a single database. If left blank, OpenMetadata ingestion attempts to scan all the databases.
+- **Driver**: Connecting to Synapse requires ODBC driver to be installed. Specify ODBC driver name in the field.
+You can download the ODBC driver from [here](https://learn.microsoft.com/en-us/sql/connect/odbc/download-odbc-driver-for-sql-server?view=sql-server-ver16). In case of docker or kubernetes deployment this driver comes out of the box with version  `ODBC Driver 18 for SQL Server`.
+
+**Authentication Mode**:
+
+- **Authentication**:
+   - The `authentication` parameter determines the method of authentication when connecting to Synapse using ODBC (Open Database Connectivity).
+   - If you select **"Active Directory Password"**, you'll need to provide the password associated with your Azure Active Directory account.
+   - Alternatively, if you choose **"Active Directory Integrated"**, the connection will use the credentials of the currently logged-in user. This mode ensures secure and seamless connections with Synapse.
+
+- **Encrypt**:
+   - The `encrypt` setting in the connection string pertains to data encryption during communication with Synapse.
+   - When enabled, it ensures that data exchanged between your application and the database is encrypted, enhancing security.
+
+- **Trust Server Certificate**:
+   - The `trustServerCertificate` option also relates to security.
+   - When set to true, your application will trust the server's SSL certificate without validation. Use this cautiously, as it bypasses certificate validation checks.
+
+- **Connection Timeout**:
+   - The `connectionTimeout` parameter specifies the maximum time (in seconds) that your application will wait while attempting to establish a connection to Synapse.
+   - If the connection cannot be established within this timeframe, an error will be raised.
+
+{% partial file="/v1.5/connectors/database/advanced-configuration.md" /%}
+
+{% /extraContent %}
+
+{% partial file="/v1.5/connectors/test-connection.md" /%}
+
+{% partial file="/v1.5/connectors/database/configure-ingestion.md" /%}
+
+{% partial file="/v1.5/connectors/ingestion-schedule-and-deploy.md" /%}
+
+{% /stepsContainer %}
+
+{% partial file="/v1.5/connectors/troubleshooting.md" /%}
+
+{% partial file="/v1.5/connectors/database/related.md" /%}

openmetadata-docs/content/v1.5.x-SNAPSHOT/connectors/database/synapse/troubleshooting.md

@@ -0,0 +1,36 @@
+---
+title: Synapse Connector Troubleshooting
+slug: /connectors/database/synapse/troubleshooting
+---
+
+# Troubleshooting
+
+Learn how to resolve the most common problems people encounter in the Synapse connector.
+
+* **Unknown error connecting with Engine [...]; An attempt to complete a transaction has failed. No corresponding transaction found. (111214) (SQLEndTran)**
+
+This is an exception you can get when trying to connect to Synapse using SQLAlchemy (the internal OpenMetadata Ingestion
+library for reaching databases).
+
+To solve this issue, you can edit your Service Connection by adding the following **Connection Argument**:
+- Key: `autocommit`
+- Value: `true`
+
+{% image
+src="/images/v1.5/connectors/synapse/autocommit.webp"
+alt="autocommit" /%}
+
+
+
+* **Cannot open server '[server name]' requested by the login. Client with IP address '[your IP]' is not allowed to access the server**
+
+This is an exception you can get when trying to connect to Synapse using SQLAlchemy (the internal OpenMetadata Ingestion library for reaching databases).
+
+
+To solve this issue, you need to add your IP address in firewall rules for your Azure SQL instance.
+
+{% image
+src="/images/v1.5/connectors/synapse/azure-firewall.webp"
+alt="azure sql firewall rules"
+caption="azure sql firewall rules" /%}
+