Skip to content

Commit

Permalink
Merge pull request #1688 from aosingh/current
Browse files Browse the repository at this point in the history 
Update Oracle profile
  • Loading branch information
@dataders
dataders authored Jul 15, 2022
2 parents 5cbc87d + b5bb324 commit 4d63db1
Showing 1 changed file with 188 additions and 72 deletions.
260 changes: 188 additions & 72 deletions website/docs/reference/warehouse-profiles/oracle-profile.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ title: "Oracle Profile"

**Maintained by:** Oracle
**Source:** [Github](https://github.com/oracle/dbt-oracle)
**Core version:** v1.0.7
**Core version:** v1.1.1
**dbt Cloud:** Not Supported
**dbt Slack channel** [#db-oracle](https://getdbt.slack.com/archives/C01PWH4TXLY)

Expand All @@ -18,9 +18,54 @@ dbt-oracle can be installed via the Python Package Index (PyPI) using pip

pip install dbt-oracle

### Configure the Python driver mode

:::info
[python-oracledb](https://oracle.github.io/python-oracledb/) is the renamed, major release of Oracle's popular cx_Oracle interface
:::

[python-oracledb](https://oracle.github.io/python-oracledb/) makes it optional to install the Oracle Client libraries.
This driver supports 2 modes

1. **Thin mode (preferred) ** : Python process directly connects to the Oracle database. This mode does not need the Oracle Client libraries
2. **Thick mode** : Python process links with the Oracle Client libraries. Some advanced Oracle database functionalities (for e.g. Advanced Queuing and Scrollable cursors) are currently available via Oracle Client libraries

It is highly recommended to use the **thin** mode as it vastly simplifies installation. You can configure the driver mode using the environment variable `ORA_PYTHON_DRIVER_TYPE`

| Driver Mode | Oracle Client libraries required? | Configuration |
|------------------------|-----------------------------------| ------------- |
| Thin | No | `ORA_PYTHON_DRIVER_TYPE=thin`|
| Thick | Yes | `ORA_PYTHON_DRIVER_TYPE=thick` |
| cx_oracle (old driver) | Yes | `ORA_PYTHON_DRIVER_TYPE=cx` |

The default value of `ORA_PYTHON_DRIVER_TYPE` is `cx`. This might change in the future as more users migrate towards the new python driver.

<Tabs
defaultValue="thin"
values={[
{ label: 'Thin', value: 'thin'},
{ label: 'Thick', value: 'thick_or_cx'}]
}>

<TabItem value="thin">

```bash
export ORA_PYTHON_DRIVER_TYPE=thin
```

</TabItem>

<TabItem value="thick_or_cx">

```bash
export ORA_PYTHON_DRIVER_TYPE=thick
# or
export ORA_PYTHON_DRIVER_TYPE=cx # default
```

### Install Oracle Instant Client libraries

To use dbt-oracle, you will need the [Oracle Instant Client libraries](https://www.oracle.com/database/technologies/instant-client.html) installed. These provide the necessary network connectivity allowing dbt-oracle to access an Oracle Database instance.
In thick mode or the old cx_oracle mode, you will need the [Oracle Instant Client libraries](https://www.oracle.com/database/technologies/instant-client.html) installed. These provide the necessary network connectivity allowing dbt-oracle to access an Oracle Database instance.

Oracle client libraries versions 21, 19, 18, 12, and 11.2 are supported where available on Linux, Windows and macOS (Intel x86). It is recommended to use the latest client possible: Oracle’s standard client-server version interoperability allows connection to both older and newer databases.

Expand Down Expand Up @@ -134,8 +179,103 @@ Note that Oracle Client versions 21c and 19c are not supported on Windows 7.
</TabItem>


</Tabs>
</TabItem>
</Tabs>

## Configure wallet for Oracle Autonomous Database in Cloud

dbt can connect to Oracle Autonomous Database (ADB) in Oracle Cloud using either TLS (Transport Layer Security) or mutual TLS (mTLS). TLS and mTLS provide enhanced security for authentication and encryption.
A database username and password is still required for dbt connections which can be configured as explained in the next section [Connecting to Oracle Database](#connecting-to-oracle-database).

<Tabs
defaultValue="tls"
values={[
{ label: 'TLS', value: 'tls'},
{ label: 'Mutual TLS', value: 'm-tls'}]
}>

<TabItem value="tls">

With TLS, dbt can connect to Oracle ADB without using a wallet. Both Thin and Thick modes of the python-oracledb driver support TLS.

:::info
In Thick mode, dbt can connect through TLS only when using Oracle Client library versions 19.14 (or later) or 21.5 (or later).
:::

Refer to Oracle documentation to [connect to an ADB instance using TLS authentication](https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/connecting-nodejs-tls.html#GUID-B3809B88-D2FB-4E08-8F9B-65A550F93A07) and the blog post [Easy wallet-less connections to Oracle Autonomous Databases in Python](https://blogs.oracle.com/opal/post/easy-way-to-connect-python-applications-to-oracle-autonomous-databases) to enable TLS for your Oracle ADB instance.
</TabItem>

<TabItem value="m-tls">

For mutual TLS connections, a wallet needs be downloaded from the OCI console and the python driver needs to be configured to use it.

#### Install the Wallet and Network Configuration Files

From the Oracle Cloud console for the database, download the wallet zip file using the `DB Connection` button. The zip contains the wallet and network configuration files.

:::warning Note
Keep wallet files in a secure location and share them only with authorized users.
:::

Unzip the wallet zip file.

<Tabs
defaultValue="thin"
values={[
{ label: 'Thin', value: 'thin'},
{ label: 'Thick', value: 'thick_or_cx'}]
}>

<TabItem value="thin">
In Thin mode, only two files from the zip are needed:

- `tnsnames.ora` - Maps net service names used for application connection strings to your database services

- `ewallet.pem` - Enables SSL/TLS connections in Thin mode. Keep this file secure

After unzipping the files in a secure directory, set the **TNS_ADMIN** and **WALLET_LOCATION** environment variables to the directory name.

```bash
export WALLET_LOCATION=/path/to/directory_containing_ewallet.pem
export WALLET_PASSWORD=***
export TNS_ADMIN=/path/to/directory_containing_tnsnames.ora
```
Optionally, if `ewallet.pem` file is encrypted using a wallet password, specify the password using environment variable **WALLET_PASSWORD**

</TabItem>

<TabItem value="thick_or_cx">
In Thick mode, the following files from the zip are needed:

- `tnsnames.ora` - Maps net service names used for application connection strings to your database services
- `sqlnet.ora` - Configures Oracle Network settings
- `cwallet.sso` - Enables SSL/TLS connections

After unzipping the files in a secure directory, set the **TNS_ADMIN** environment variable to that directory name.

```bash
export TNS_ADMIN=/path/to/directory_containing_tnsnames.ora
```

Next, edit the `sqlnet.ora` file to point to the wallet directory.

<File name='sqlnet.ora'>

```text
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="/path/to/wallet/directory")))
SSL_SERVER_DN_MATCH=yes
```

</File>

</TabItem>
</Tabs>

</TabItem>
</Tabs>


## Connecting to Oracle Database

Define the following mandatory parameters as environment variables and refer them in the connection profile using [env_var](https://docs.getdbt.com/reference/dbt-jinja-functions/env_var) jinja function. Optionally, you can also define these directly in the `profiles.yml` file, but this is not recommended
Expand All @@ -149,7 +289,7 @@ export DBT_ORACLE_SCHEMA=<username>
Starting with `dbt-oracle==1.0.2`, it is **optional** to set the database name

```bash
export DBT_ORACLE_DATABASE=ga01d78d2ecd5f1_db202112221108
export DBT_ORACLE_DATABASE=example_db2022adb
```

If database name is not set, adapter will retrieve it using the following query.
Expand All @@ -161,47 +301,13 @@ SELECT SYS_CONTEXT('userenv', 'DB_NAME') FROM DUAL
An Oracle connection profile for dbt can be set using any one of the following methods

<Tabs
defaultValue="database_hostname"
defaultValue="tns_net_service_name"
values={[
{ label: 'Using Database hostname', value: 'database_hostname'},
{ label: 'Using TNS net service name', value: 'tns_net_service_name'},
{ label: 'Using Connect string', value:'connect_string'}]
{ label: 'Using TNS alias', value: 'tns_net_service_name'},
{ label: 'Using Connect string', value:'connect_string'},
{ label: 'Using Database hostname', value: 'database_hostname'}]
}>

<TabItem value="database_hostname">

To connect using the database hostname or IP address, you need to specify the following
- host
- port (1521 or 1522)
- protocol (tcp or tcps)
- service

```bash
export DBT_ORACLE_HOST=adb.us-ashburn-1.oraclecloud.com
export DBT_ORACLE_SERVICE=ga01d78d2ecd5f1_db202112221108_high.adb.oraclecloud.com
```

<File name='~/.dbt/profiles.yml'>

```yaml
dbt_test:
target: "{{ env_var('DBT_TARGET', 'dev') }}"
outputs:
dev:
type: oracle
user: "{{ env_var('DBT_ORACLE_USER') }}"
pass: "{{ env_var('DBT_ORACLE_PASSWORD') }}"
protocol: "tcps"
host: "{{ env_var('DBT_ORACLE_HOST') }}"
port: 1522
service: "{{ env_var('DBT_ORACLE_SERVICE') }}"
database: "{{ env_var('DBT_ORACLE_DATABASE') }}"
schema: "{{ env_var('DBT_ORACLE_SCHEMA') }}"
threads: 4
```
</File>
</TabItem>

<TabItem value="tns_net_service_name">

The `tnsnames.ora` file is a configuration file that contains network service names mapped to connect descriptors.
Expand All @@ -210,18 +316,21 @@ The directory location of `tnsnames.ora` file can be specified using `TNS_ADMIN`
<File name="tnsnames.ora">

```text
net_service_name=
(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=dbhost.example.com)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=orclpdb1)))
db2022adb_high = (description = (
address=(protocol=tcps)
(port=1522)
(host=adb.example.oraclecloud.com))
(connect_data=(service_name=example_high.adb.oraclecloud.com))
(security=(ssl_server_cert_dn="CN=adb.example.oraclecloud.com,
OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US")))
```

</File>

The `net_service_name` can be defined as environment variable and referred in `profiles.yml`
The TNS alias `db2022adb_high` can be defined as environment variable and referred in `profiles.yml`

```bash
export DBT_ORACLE_TNS_NAME=net_service_name
export DBT_ORACLE_TNS_NAME=db2022adb_high
```

<File name='~/.dbt/profiles.yml'>
Expand Down Expand Up @@ -251,7 +360,10 @@ The connection string identifies which database service to connect to. It can be
- A Net Service Name mapping to a connect descriptor

```bash
export DBT_ORACLE_CONNECT_STRING="(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dbhost.example.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=orclpdb1)))"
export DBT_ORACLE_CONNECT_STRING="(description=(address=(protocol=tcps)(port=1522)
(host=adb.example.oraclecloud.com))(connect_data=(service_name=example_high.adb.oraclecloud.com))
(security=(ssl_server_cert_dn=\"CN=adb.example.oraclecloud.com,
OU=Oracle BMCS US,O=Oracle Corporation,L=Redwood City,ST=California,C=US\")))"
```

<File name='~/.dbt/profiles.yml'>
Expand All @@ -272,38 +384,42 @@ dbt_test:
</File>
</TabItem>

</Tabs>

## Connecting to Oracle Autonomous Database in Cloud

To enable connection to Oracle Autonomous Database in Oracle Cloud, a wallet needs be downloaded from the cloud, and cx_Oracle needs to be configured to use it. The wallet gives mutual TLS which provides enhanced security for authentication and encryption. A database username and password is still required for your application connections.

### Install the Wallet and Network Configuration Files

From the Oracle Cloud console for the database, download the wallet zip file. It contains the wallet and network configuration files. Note: keep wallet files in a secure location and share them only with authorized users.

Unzip the wallet zip file. For cx_Oracle, only these files from the zip are needed:
<TabItem value="database_hostname">

- `tnsnames.ora` - Maps net service names used for application connection strings to your database services
- `sqlnet.ora` - Configures Oracle Network settings
- `cwallet.sso` - Enables SSL/TLS connections
To connect using the database hostname or IP address, you need to specify the following
- host
- port (1521 or 1522)
- protocol (tcp or tcps)
- service

After downloading the wallet, put the unzipped wallet files in a secure directory and set the TNS_ADMIN environment variable to that directory name. Next, edit the sqlnet.ora file to point to the wallet directory.
```bash
export DBT_ORACLE_HOST=adb.example.oraclecloud.com
export DBT_ORACLE_SERVICE=example_high.adb.oraclecloud.com
```

<File name='sqlnet.ora'>
<File name='~/.dbt/profiles.yml'>

```text
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="/path/to/wallet/directory")))
SSL_SERVER_DN_MATCH=yes
```yaml
dbt_test:
target: "{{ env_var('DBT_TARGET', 'dev') }}"
outputs:
dev:
type: oracle
user: "{{ env_var('DBT_ORACLE_USER') }}"
pass: "{{ env_var('DBT_ORACLE_PASSWORD') }}"
protocol: "tcps"
host: "{{ env_var('DBT_ORACLE_HOST') }}"
port: 1522
service: "{{ env_var('DBT_ORACLE_SERVICE') }}"
database: "{{ env_var('DBT_ORACLE_DATABASE') }}"
schema: "{{ env_var('DBT_ORACLE_SCHEMA') }}"
threads: 4
```

</File>
</TabItem>

:::info TLS v/s mTLS

If you have enabled TLS connections on your Database instance then dbt can connect using only database username, password and the Oracle Net connect name given in the unzipped tnsnames.ora file.

:::
</Tabs>


## Supported Features
Expand Down

0 comments on commit 4d63db1

Please sign in to comment.