Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Broken links corrected #14489

Merged
merged 1 commit into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def __init__(
):
# ssl_verify is typed as a `bool` in TableauServerConnection
# However, it is passed as `verify=self.ssl_verify` in each `requests` call.
# In requests (https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification)
# In requests (https://requests.readthedocs.io/en/latest/user/advanced.html?highlight=ssl#ssl-cert-verification)
# the param can be None, False to ignore HTTPS certs or a string with the path to the cert.
self._client = TableauServerConnection(
config_json=config,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ In order to integrate SSL in the Metadata Ingestion Config, the user will have t
There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced.html?highlight=ssl#ssl-cert-verification).

{% /codeInfo %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ In order to integrate SSL in the Metadata Ingestion Config, the user will have t
There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced.html?highlight=ssl#ssl-cert-verification).

{% /extraContent %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ that decouples how OpenMetadata communicates with the Orchestrator, as different
calls and data to be sent.

- You can find the `PipelineServiceClient` abstraction [here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/util/PipelineServiceClient.java),
- And the `AirflowRESTClient` implementation [here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/airflow/AirflowRESTClient.java).
- And the `AirflowRESTClient` implementation [here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/clients/pipeline/airflow/AirflowRESTClient.java).

The clients that implement the abstractions from the `PipelineServiceClient` are merely a translation layer between the
information received in the shape of an `IngestionPipeline` Entity, and the specific requirements of each Orchestrator.
Expand Down Expand Up @@ -284,7 +284,7 @@ pipelineServiceClient.deployPipeline(ingestionPipeline);
```

Then, the actual deployment logic is handled by the class implementing the Pipeline Service Client. For this example,
it will be the [AirflowRESTClient](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/airflow/AirflowRESTClient.java).
it will be the [AirflowRESTClient](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/clients/pipeline/airflow/AirflowRESTClient.java).

First, let's see what it is needed to instantiate the Airflow REST Client:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,12 @@ as well). You might also need to validate if the query logs are available in the

You can check the queries being used here:

- [BigQuery](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L428)
- [Snowflake](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L197)
- [MSSQL](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L350)
- [Redshift](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L18)
- [Clickhouse](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L376)
- [Postgres](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/utils/sql_queries.py#L467)
- [BigQuery](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/bigquery/queries.py)
- [Snowflake](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/snowflake/queries.py)
- [MSSQL](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/mssql/queries.py)
- [Redshift](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/redshift/queries.py)
- [Clickhouse](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/clickhouse/queries.py)
- [Postgres](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/src/metadata/ingestion/source/database/postgres/queries.py)

By default, we apply a result limit of 1000 records. You might also need to increase that for databases with big volumes
of queries.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ the steps to create a YAML configuration able to connect to the source,
process the Entities if needed, and reach the OpenMetadata server.

The workflow is modeled around the following
[JSON Schema](https://github.com/open-metadata/OpenMetadatablob/main/openmetadata-spec/src/main/resources/json/schema/metadataIngestion/workflow.json)
[JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/metadataIngestion/mlmodelServiceMetadataPipeline.json)

### 1. Define the YAML Config

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ pip3 install "openmetadata-ingestion[sagemaker]"
## Metadata Ingestion

All connectors are defined as JSON Schemas.
[Here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/services/connections/mlmodel/sagemakerConnection.json)
[Here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/services/connections/mlmodel/sageMakerConnection.json)
you can find the structure to create a connection to Sagemaker.

In order to create and run a Metadata Ingestion workflow, we will follow
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,15 +55,15 @@ pip3 install "openmetadata-ingestion[sagemaker]"
## Metadata Ingestion

All connectors are defined as JSON Schemas.
[Here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/services/connections/mlmodel/sagemakerConnection.json)
[Here](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/services/connections/mlmodel/sageMakerConnection.json)
you can find the structure to create a connection to Sagemaker.

In order to create and run a Metadata Ingestion workflow, we will follow
the steps to create a YAML configuration able to connect to the source,
process the Entities if needed, and reach the OpenMetadata server.

The workflow is modeled around the following
[JSON Schema](https://github.com/open-metadata/OpenMetadatablob/main/openmetadata-spec/src/main/resources/json/schema/metadataIngestion/workflow.json)
[JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/metadataIngestion/mlmodelServiceMetadataPipeline.json)

### 1. Define the YAML Config

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,5 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ OM_AUTH_JWT_TOKEN=
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

## 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,5 @@ global:


{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ authorizerConfiguration:


{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

## 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,5 @@ global:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,5 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ global:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ AUTHENTICATION_CALLBACK_URL=http://openmetadata:8585/callback


{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

### 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,5 @@ global:


{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,5 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

## 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,5 +51,5 @@ global:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,5 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

## 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,5 +50,5 @@ global:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ authorizerConfiguration:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}


Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}

## 2. Start Docker
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ global:
```

{% note noteType="Tip" %}
Follow [this guide](/how-to-guides/feature-configurations/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
Follow [this guide](/how-to-guides/admin-guide/bots) to configure the `ingestion-bot` credentials for ingesting data using Connectors.
{% /note %}
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,4 @@ In this doc, we have seen the role that OpenMetadata serves from a Machine Learn

We have shown how to use the Python API to enrich the models' metadata and add the lineage information with its related Entities and how the versioning will respond to changes.

For further information on the properties, do not hesitate to review the [JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/data/mlmodel.json) and for examples and usages with the Python API, you can take a look at our [integration tests](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/tests/integration/ometa/test_ometa_model_api.py).
For further information on the properties, do not hesitate to review the [JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-spec/src/main/resources/json/schema/entity/data/mlmodel.json) and for examples and usages with the Python API, you can take a look at our [integration tests](https://github.com/open-metadata/OpenMetadata/blob/main/ingestion/tests/integration/ometa/test_ometa_mlmodel_api.py).
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ In order to integrate SSL in the Metadata Ingestion Config, the user will have t
There are a couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
Find more details in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
Find more details in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced.html?highlight=ssl#ssl-cert-verification).

{% partial file="/v1.1/connectors/database/advanced-configuration.md" /%}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -512,7 +512,7 @@ In order to integrate SSL in the Metadata Ingestion Config, the user will have t
There are couple of types of SSL modes that redshift supports which can be added to ConnectionArguments, they are as follows:
- **false**: In order to disable SSL verification, set the `verify` parameter to `False`.
- **\<path-to-crt\>**: To use self-signed certificates, specify a path to the certificate in `verify` parameter.
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced/#ssl-cert-verification).
More details can be found in [the Python requests library documentation](https://requests.readthedocs.io/en/latest/user/advanced.html?highlight=ssl#ssl-cert-verification).

{% /codeInfo %}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ pipelineServiceClient.deployPipeline(ingestionPipeline);
```

Then, the actual deployment logic is handled by the class implementing the Pipeline Service Client. For this example,
it will be the [AirflowRESTClient](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/airflow/AirflowRESTClient.java).
it will be the [AirflowRESTClient](https://github.com/open-metadata/OpenMetadata/blob/main/openmetadata-service/src/main/java/org/openmetadata/service/clients/pipeline/airflow/AirflowRESTClient.java).

First, let's see what it is needed to instantiate the Airflow REST Client:

Expand Down
Loading
Loading