Skip to content

Commit

Permalink
add stich docs (#18666)
Browse files Browse the repository at this point in the history
  • Loading branch information
harshsoni2024 authored and harshach committed Nov 24, 2024
1 parent 66aad30 commit b39e792
Show file tree
Hide file tree
Showing 8 changed files with 205 additions and 0 deletions.
4 changes: 4 additions & 0 deletions openmetadata-docs/content/v1.6.x-SNAPSHOT/collate-menu.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,10 @@ site_menu:

- category: Connectors / Pipeline / DataFactory / Run Externally
url: /connectors/pipeline/datafactory/yaml
- category: Connectors / Pipeline / Stitch
url: /connectors/pipeline/stitch
- category: Connectors / Pipeline / Stitch / Run Externally
url: /connectors/pipeline/stitch/yaml
- category: Connectors / Pipeline / Databricks Pipeline
url: /connectors/pipeline/databricks-pipeline
- category: Connectors / Pipeline / Databricks Pipeline / Run Externally
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
---
title: Stitch
slug: /connectors/pipeline/stitch
collate: true
---

{% connectorDetailsHeader
name="Stitch"
stage="PROD"
platform="Collate"
availableFeatures=["Pipelines", "Lineage"]
unavailableFeatures=["Owners", "Tags", "Pipeline Status"]
/ %}


In this section, we provide guides and references to use the Stitch connector.

Configure and schedule Stitch metadata and profiler workflows from the OpenMetadata UI:

- [Requirements](#requirements)
- [Metadata Ingestion](#metadata-ingestion)

{% partial file="/v1.6/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/pipeline/stitch/yaml"} /%}

## Requirements

To extract metadata from Stitch, User first need to crate API crednetials:
- `Token`: Token to access Stitch metadata.


## Metadata Ingestion

{% partial
file="/v1.6/connectors/metadata-ingestion-ui.md"
variables={
connector: "Stitch",
selectServicePath: "/images/v1.6/connectors/stitch/select-service.png",
addNewServicePath: "/images/v1.6/connectors/stitch/add-new-service.png",
serviceConnectionPath: "/images/v1.6/connectors/stitch/service-connection.png",
}
/%}

{% stepsContainer %}
{% extraContent parentTagName="stepsContainer" %}

#### Connection Details

- **hostPort**: This parameter specifies the network location where your Stitch instance is accessible, combining both the hostname. It is based on the account and region where user has hosted his/her data pipelines. More about this you can check [here](https://www.stitchdata.com/docs/developers/import-api/api#base-urls)

- **token**: Token to get access to Stitch metadata. This token is created by user after logging into stitch console. More about this, please check [here](https://www.stitchdata.com/docs/developers/import-api/guides/quick-start#obtain-api-credentials)


{% /extraContent %}

{% partial file="/v1.6/connectors/test-connection.md" /%}

{% partial file="/v1.6/connectors/pipeline/configure-ingestion.md" /%}

{% partial file="/v1.6/connectors/ingestion-schedule-and-deploy.md" /%}

{% /stepsContainer %}

By successfully completing these steps, the lineage information for the service will be displayed.

{% partial file="/v1.6/connectors/troubleshooting.md" /%}
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
---
title: Run the Stitch Connector Externally
slug: /connectors/pipeline/stitch/yaml
collate: true
---

{% connectorDetailsHeader
name="Stitch"
stage="PROD"
platform="Collate"
availableFeatures=["Pipelines", "Lineage"]
unavailableFeatures=["Owners", "Tags", "Pipeline Status"]
/ %}



In this section, we provide guides and references to use the Stitch connector.

Configure and schedule Stitch metadata and profiler workflows from the OpenMetadata UI:

- [Requirements](#requirements)
- [Metadata Ingestion](#metadata-ingestion)

{% partial file="/v1.6/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/pipeline/stitch/yaml"} /%}

## Requirements

To extract metadata from Stitch, User first need to crate API crednetials:
- `Token`: Token to access Stitch metadata.


### Python Requirements

{% partial file="/v1.6/connectors/python-requirements.md" /%}

To run the Stitch ingestion, you will need to install:

```bash
pip3 install "openmetadata-ingestion[stitch]"
```

## Metadata Ingestion

All connectors are defined as JSON Schemas.
[Here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/services/connections/pipeline/stitchConnection.json)
you can find the structure to create a connection to Stitch.

In order to create and run a Metadata Ingestion workflow, we will follow
the steps to create a YAML configuration able to connect to the source,
process the Entities if needed, and reach the OpenMetadata server.

The workflow is modeled around the following
[JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/metadataIngestion/workflow.json)

### 1. Define the YAML Config

This is a sample config for Stitch:

{% codePreview %}

{% codeInfoContainer %}

#### Source Configuration - Service Connection

{% codeInfo srNumber=1 %}

**hostPort**: The hostname or IP address with the REST API enabled eg.`https://api.stitchdata.com`

{% /codeInfo %}

{% codeInfo srNumber=2 %}

**token**: Token to get access to Stitch metadata.

{% /codeInfo %}


{% partial file="/v1.6/connectors/yaml/pipeline/source-config-def.md" /%}

{% partial file="/v1.6/connectors/yaml/ingestion-sink-def.md" /%}

{% partial file="/v1.6/connectors/yaml/workflow-config-def.md" /%}

{% /codeInfoContainer %}

{% codeBlock fileName="filename.yaml" %}


```yaml {% isCodeBlock=true %}
source:
type: stitch
serviceName: stitch_data
serviceConnection:
config:
type: Stitch
```
```yaml {% srNumber=1 %}
hostPort: "https://api.stitchdata.com"
```
```yaml {% srNumber=2 %}
token: "token"
```
{% partial file="/v1.6/connectors/yaml/pipeline/source-config.md" /%}
{% partial file="/v1.6/connectors/yaml/ingestion-sink.md" /%}
{% partial file="/v1.6/connectors/yaml/workflow-config.md" /%}
{% /codeBlock %}
{% /codePreview %}
{% partial file="/v1.6/connectors/yaml/ingestion-cli.md" /%}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Stitch

In this section, we provide guides and references to use the Stitch connector.

## Requirements:

To extract metadata from Stitch, User first need to crate API crednetials:
- `Token`: Token to access Stitch metadata.

## Connection Details

$$section
### Host Port $(id="hostPort")
This parameter specifies the network location where your Stitch instance is accessible, combining both the hostname.
It is based on the account and region where user has hosted his/her data pipelines. More about this you can check [here](https://www.stitchdata.com/docs/developers/import-api/api#base-urls)
$$

$$section
### Token $(id="token")
Token to get access to Stitch metadata. This token is created by user after logging into stitch console. More about this, please check [here](https://www.stitchdata.com/docs/developers/import-api/guides/quick-start#obtain-api-credentials)
$$
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ class ServiceUtilClassBase {
MlModelServiceType.VertexAI,
PipelineServiceType.Matillion,
PipelineServiceType.DataFactory,
PipelineServiceType.Stitch,
];

DatabaseServiceTypeSmallCase = this.convertEnumToLowerCase<
Expand Down

0 comments on commit b39e792

Please sign in to comment.