Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update docs for paddle and snowflake fdw new version #331

Merged
merged 10 commits into from
Aug 15, 2024
Merged
2 changes: 1 addition & 1 deletion .github/workflows/release_wasm_fdw.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ jobs:
...
);

For more details, please visit https://supabase.github.io/wrappers/.
For more details, please visit https://fdw.dev.
EOF

- name: Create release
Expand Down
Binary file added docs/assets/fdw-dark.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/assets/fdw-light.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 3 additions & 3 deletions docs/catalog/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,9 @@ hide:

Wasm wrappers can be installed directly from GitHub or any external source.

See [Developing a Wasm Wrapper](/wrappers/guides/create-wasm-wrapper/) for instructions on how to build and develop your own.
See [Developing a Wasm Wrapper](../guides/create-wasm-wrapper.md) for instructions on how to build and develop your own.

| Integration | Developer | Docs | Source |
| ----------- | :------------------------------: | :----------------------------------: | :------------------------------------------------------------------------------------: |
| Paddle | [Supabase](https://supabase.com) | [Link](/wrappers/catalog/paddle/) | [Link](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/paddle_fdw) |
| Snowflake | [Supabase](https://supabase.com) | [Link](/wrappers/catalog/snowflake/) | [Link](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/snowflake_fdw) |
| Paddle | [Supabase](https://supabase.com) | [Link](paddle.md) | [Link](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/paddle_fdw) |
| Snowflake | [Supabase](https://supabase.com) | [Link](snowflake.md) | [Link](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/snowflake_fdw) |
27 changes: 18 additions & 9 deletions docs/catalog/paddle.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ The Paddle API uses JSON formatted data, please refer to [Paddle docs](https://d

| Version | Wasm Package URL | Checksum |
| ------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| 0.1.1 | `https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.1/paddle_fdw.wasm` | `c5ac70bb2eef33693787b7d4efce9a83cde8d4fa40889d2037403a51263ba657` |
| 0.1.0 | `https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.0/paddle_fdw.wasm` | `7d0b902440ac2ef1af85d09807145247f14d1d8fd4d700227e5a4d84c8145409` |

## Preparation
Expand Down Expand Up @@ -81,10 +82,10 @@ We need to provide Postgres with the credentials to access Paddle, and any addit
create server paddle_server
foreign data wrapper wasm_wrapper
options (
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.0/paddle_fdw.wasm',
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.1/paddle_fdw.wasm',
fdw_package_name 'supabase:paddle-fdw',
fdw_package_version '0.1.0',
fdw_package_checksum '7d0b902440ac2ef1af85d09807145247f14d1d8fd4d700227e5a4d84c8145409',
fdw_package_version '0.1.1',
fdw_package_checksum 'c5ac70bb2eef33693787b7d4efce9a83cde8d4fa40889d2037403a51263ba657',
api_url 'https://sandbox-api.paddle.com', -- Use https://api.paddle.com for live account
api_key_id '<key_ID>' -- The Key ID from above.
);
Expand All @@ -96,17 +97,25 @@ We need to provide Postgres with the credentials to access Paddle, and any addit
create server paddle_server
foreign data wrapper wasm_wrapper
options (
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.0/paddle_fdw.wasm',
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_paddle_fdw_v0.1.1/paddle_fdw.wasm',
fdw_package_name 'supabase:paddle-fdw',
fdw_package_version '0.1.0',
fdw_package_checksum '7d0b902440ac2ef1af85d09807145247f14d1d8fd4d700227e5a4d84c8145409',
fdw_package_version '0.1.1',
fdw_package_checksum 'c5ac70bb2eef33693787b7d4efce9a83cde8d4fa40889d2037403a51263ba657',
api_url 'https://sandbox-api.paddle.com', -- Use https://api.paddle.com for live account
api_key 'bb4e69088ea07a98a90565ac610c63654423f8f1e2d48b39b5'
);
```

Note the `fdw_package_*` options are required, which specify the Wasm package metadata. You can get the available package version list from [above](#available-versions).

### Create a schema

We recommend creating a schema to hold all the foreign tables:

```sql
create schema if not exists paddle;
```

## Creating Foreign Tables

The Paddle Wrapper supports data reads and writes from Paddle.
Expand All @@ -118,7 +127,7 @@ The Paddle Wrapper supports data reads and writes from Paddle.
For example:

```sql
create foreign table paddle_customers (
create foreign table paddle.customers (
id text,
name text,
email text,
Expand Down Expand Up @@ -161,7 +170,7 @@ The full list of foreign table options are below:
This FDW supports `where` clause pushdown with `id` as the filter. For example,

```sql
select * from paddle_customers where id = 'ctm_01hymwgpkx639a6mkvg99563sp';
select * from paddle.customers where id = 'ctm_01hymwgpkx639a6mkvg99563sp';
```

## Examples
Expand All @@ -173,7 +182,7 @@ Below are Some examples on how to use Paddle foreign tables.
This example will create a "foreign table" inside your Postgres database and query its data. First, we can create a schema to hold all the Paddle foreign tables.

```sql
create schema paddle;
create schema if not exists paddle;
```

Then create the foreign table and query it, for example:
Expand Down
28 changes: 20 additions & 8 deletions docs/catalog/snowflake.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ The Snowflake Wrapper is a WebAssembly(Wasm) foreign data wrapper which allows y

| Version | Wasm Package URL | Checksum |
| ------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| 0.1.1 | `https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.1/snowflake_fdw.wasm` | `7aaafc7edc1726bc93ddc04452d41bda9e1a264a1df2ea9bf1b00b267543b860` |
| 0.1.0 | `https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.0/snowflake_fdw.wasm` | `2fb46fd8afa63f3975dadf772338106b609b131861849356e0c09dde032d1af8` |

## Preparation
Expand Down Expand Up @@ -81,10 +82,10 @@ We need to provide Postgres with the credentials to connect to Snowflake, and an
create server snowflake_server
foreign data wrapper wasm_wrapper
options (
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.0/snowflake_fdw.wasm',
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.1/snowflake_fdw.wasm',
fdw_package_name 'supabase:snowflake-fdw',
fdw_package_version '0.1.0',
fdw_package_checksum '2fb46fd8afa63f3975dadf772338106b609b131861849356e0c09dde032d1af8',
fdw_package_version '0.1.1',
fdw_package_checksum '7aaafc7edc1726bc93ddc04452d41bda9e1a264a1df2ea9bf1b00b267543b860',
account_identifier 'MYORGANIZATION-MYACCOUNT',
user 'MYUSER',
public_key_fingerprint 'SizgPofeFX0jwC8IhbOfGFyOggFgo8oTOS1uPLZhzUQ=',
Expand All @@ -98,10 +99,10 @@ We need to provide Postgres with the credentials to connect to Snowflake, and an
create server snowflake_server
foreign data wrapper wasm_wrapper
options (
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.0/snowflake_fdw.wasm',
fdw_package_url 'https://github.com/supabase/wrappers/releases/download/wasm_snowflake_fdw_v0.1.1/snowflake_fdw.wasm',
fdw_package_name 'supabase:snowflake-fdw',
fdw_package_version '0.1.0',
fdw_package_checksum '2fb46fd8afa63f3975dadf772338106b609b131861849356e0c09dde032d1af8',
fdw_package_version '0.1.1',
fdw_package_checksum '7aaafc7edc1726bc93ddc04452d41bda9e1a264a1df2ea9bf1b00b267543b860',
account_identifier 'MYORGANIZATION-MYACCOUNT',
user 'MYUSER',
public_key_fingerprint 'SizgPofeFX0jwC8IhbOfGFyOggFgo8oTOS1uPLZhzUQ=',
Expand All @@ -111,6 +112,14 @@ We need to provide Postgres with the credentials to connect to Snowflake, and an

Note the `fdw_package_*` options are required, which specify the Wasm package metadata. You can get the available package version list from [above](#available-versions).

### Create a schema

We recommend creating a schema to hold all the foreign tables:

```sql
create schema if not exists snowflake;
```

## Creating Foreign Tables

The Snowflake Wrapper supports data reads and writes from Snowflake.
Expand All @@ -122,7 +131,7 @@ The Snowflake Wrapper supports data reads and writes from Snowflake.
For example:

```sql
create foreign table snowflake_mytable (
create foreign table snowflake.mytable (
id bigint,
name text,
num numeric,
Expand Down Expand Up @@ -161,6 +170,9 @@ Some examples on how to use Snowflake foreign tables.
Let's prepare the source table in Snowflake first:

```sql
-- Create a database
create database if not exists mydatabase;

-- Run below SQLs on Snowflake to create source table
create table mydatabase.public.mytable (
id number(38,0),
Expand All @@ -182,7 +194,7 @@ values (43, 'bar', 56.78, '2024-05-19', '2024-05-19 12:34:56');
This example will create a "foreign table" inside your Postgres database and query its data. First, we can create a schema to hold all the Snowflake foreign tables.

```sql
create schema snowflake;
create schema if not exists snowflake;
```

Then create the foreign table and query it, for example:
Expand Down
8 changes: 4 additions & 4 deletions docs/catalog/wasm/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ Foreign data wrappers built with Wasm which can be used on Supabase platform.

Supported by [Supabase](https://www.supabase.com)

:octicons-tag-24: [v0.1.0](https://github.com/supabase/wrappers/releases/tag/wasm_paddle_fdw_v0.1.0) &nbsp;
:octicons-code-24: [source](https://github.com/supabase/wrappers/tree/wasm_paddle_fdw_v0.1.0/wasm-wrappers/fdw/paddle_fdw) &nbsp;
:octicons-tag-24: [v0.1.1](https://github.com/supabase/wrappers/releases/tag/wasm_paddle_fdw_v0.1.1) &nbsp;
:octicons-code-24: [source](https://github.com/supabase/wrappers/tree/wasm_paddle_fdw_v0.1.1/wasm-wrappers/fdw/paddle_fdw) &nbsp;
:material-file-document: [docs](../paddle.md)

- :simple-webassembly: &nbsp; **[Snowflake](../snowflake.md)**
Expand All @@ -33,8 +33,8 @@ Foreign data wrappers built with Wasm which can be used on Supabase platform.

Supported by [Supabase](https://www.supabase.com)

:octicons-tag-24: [v0.1.0](https://github.com/supabase/wrappers/releases/tag/wasm_snowflake_fdw_v0.1.0) &nbsp;
:octicons-code-24: [source](https://github.com/supabase/wrappers/tree/wasm_snowflake_fdw_v0.1.0/wasm-wrappers/fdw/snowflake_fdw) &nbsp;
:octicons-tag-24: [v0.1.1](https://github.com/supabase/wrappers/releases/tag/wasm_snowflake_fdw_v0.1.1) &nbsp;
:octicons-code-24: [source](https://github.com/supabase/wrappers/tree/wasm_snowflake_fdw_v0.1.1/wasm-wrappers/fdw/snowflake_fdw) &nbsp;
:material-file-document: [docs](../snowflake.md)

</div>
2 changes: 1 addition & 1 deletion docs/contributing/native.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ To know more about FDW development, please visit the [Wrappers documentation](ht

## Basic usage

These steps outline how to use the a demo FDW [HelloWorldFdw](./wrappers/src/fdw/helloworld_fdw), which only outputs a single line of fake data:
These steps outline how to use the a demo FDW [HelloWorldFdw](https://github.com/supabase/wrappers/tree/main/wrappers/src/fdw/helloworld_fdw), which only outputs a single line of fake data:

1. Clone this repo

Expand Down
4 changes: 2 additions & 2 deletions docs/guides/create-wasm-wrapper.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ env:

## Install your FDW

You can install the FDW on any Postgres [platform that supports](/wrappers/#supported-platforms) Wrappers `>=0.4.1`.
You can install the FDW on any Postgres [platform that supports](../index.md#supported-platforms) Wrappers `>=0.4.1`.

<details>

Expand All @@ -103,7 +103,7 @@ where name = 'wrappers';

### Enable Wrappers

Enable the Wrappers extension and initialize the Wasm FDW on any platform that [supports Wrappers](/wrappers/#supported-platforms):
Enable the Wrappers extension and initialize the Wasm FDW on any platform that [supports Wrappers](../index.md#supported-platforms):

```sql
create extension if not exists wrappers with schema extensions;
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/native-wasm.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

Since v0.4.0, Wrappers supports WebAssembly (Wasm) FDWs. Anyone can develop a Wasm Wrapper. Wasm foreign data wrappers are dynamically loaded on the first query and then cached locally, they can be installed directly from remotes like GitHub and S3.

Check out [Developing a Wrapper](../create-wasm-wrapper) to develop your own Wasm Wrapper.
Check out [Developing a Wrapper](create-wasm-wrapper.md) to develop your own Wasm Wrapper.

## Native Wrappers

Expand Down
63 changes: 60 additions & 3 deletions docs/guides/wasm-advanced.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Developing a Wasm Wrapper (Advanced)

If you followed the [Quick Start](../create-wasm-wrapper), you should have:
If you followed the [Quick Start](create-wasm-wrapper.md), you should have:

1. Built a GitHub FDW
2. Installed it on Postgres
Expand All @@ -10,7 +10,7 @@ This guide will show you how to develop the Wasm Wrapper locally to iterate fast

## Install pre-requisites

If you haven't already completed the [Quick Start](../create-wasm-wrapper), do that first. For local development:
If you haven't already completed the [Quick Start](create-wasm-wrapper.md), do that first. For local development:

- install the [Supabase CLI](https://supabase.com/docs/guides/cli/getting-started) (version `>= 1.187.10` is needed).
- install [Docker](https://www.docker.com/get-started/)
Expand Down Expand Up @@ -248,6 +248,63 @@ from
google.sheets
```

## Considerations

### Version compatibility

The Wasm FDW (guest) runs inside a Wasm runtime (host) which is provided by the [Wrappers Wasm FDW framework](https://github.com/supabase/wrappers/tree/main/wrappers/src/fdw/wasm_fdw). The guest and host versions need to be compatible. We can define the required host version in the guest's `host_version_requirement()` function like below:

```rust
impl Guest for ExampleFdw {
fn host_version_requirement() -> String {
"^0.1.0".to_string()
}
}
```

Both guest and host are using [Semantic Versioning](https://docs.rs/semver/latest/semver/enum.Op.html). The above code means the guest is compatible with host version greater or equal `0.1.0` but less than `0.2.0`. If the version isn't comatible, the Wasm FDW cannot run on that version of host.

All the available host versions are listed [here](https://github.com/supabase/wrappers/blob/main/wrappers/src/fdw/wasm_fdw/README.md). When you develop your own Wasm FDW, always choose compatible host version properly.

### Security

!!! warning
Never use untrusted Wasm FDW on your database.

Although we have implemented security measures and limited the Wasm runtime environment to a minimal interface, ultimately you are responsible for your data. Never install a Wasm FDW from untrusted source. Always use official sources, like [Supabase Wasm FDW](../catalog/wasm/index.md), or sources over which you have full visibility and control.

### Performance

The Wasm package will be dynamically downloaded and loaded to run on Postgres, so you should make sure the Wasm FDW is small to improve performance. Always build your project in `release` mode using the profile specified in the `Cargo.toml` file:

```toml
[profile.release]
strip = "debuginfo"
lto = true
```

```bash
# build in release mode and target wasm32-unknown-unknown
cargo component build --release --target wasm32-unknown-unknown
```

### Automation

If you host source code on GitHub, the building and release process can be automated, take a look at [the example CI workflow file](https://github.com/supabase-community/postgres-wasm-fdw/blob/main/.github/workflows/release_wasm_fdw.yml) for more details.

## Limitations

The Wasm FDW currently only supports data sources which have HTTP(s) based JSON API, other sources such like TCP based DBMS or local files are not supported.

Another limitation is that many 3rd-party Rust libraries don't support `wasm32-unknown-unknown` target, we cannot use them in the Wasm FDW project.

## Wrap up

When you're ready, you can follow the [Release process](/wrappers/guides/create-wasm-wrapper/#release-the-wasm-fdw-package) in the quickstart guide to release a new version of your wrapper.
When you're ready, you can follow the [Release process](../guides/create-wasm-wrapper.md#release-the-wasm-fdw-package) in the quickstart guide to release a new version of your wrapper.

## More examples

Some other Wasm foreign data wrapper projects developed by Supabase team:

- [Snowflake Wasm FDW](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/snowflake_fdw)
- [Paddle Wasm FDW](https://github.com/supabase/wrappers/tree/main/wasm-wrappers/fdw/paddle_fdw)
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ returns

Postgres FDWs introduce the concept of a "remote server" and "foreign table":

![FDW](assets/fdw-light.png)
![FDW](assets/fdw-dark.png)

### Remote servers

Expand Down
9 changes: 7 additions & 2 deletions mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ repo_name: supabase/wrappers
repo_url: https://github.com/supabase/wrappers
edit_uri: edit/main/docs/

not_in_nav: |
tags.md

nav:
- Home:
- 'index.md'
Expand Down Expand Up @@ -54,6 +57,8 @@ theme:
logo: 'assets/wrappers.svg'
homepage: https://supabase.github.io/wrappers
features:
- content.code.annotate
- content.code.copy
- navigation.expand
- navigation.tabs
- navigation.tabs.sticky
Expand Down Expand Up @@ -96,5 +101,5 @@ markdown_extensions:
- pymdownx.tasklist
- admonition
- pymdownx.emoji:
emoji_index: !!python/name:materialx.emoji.twemoji
emoji_generator: !!python/name:materialx.emoji.to_svg
emoji_index: !!python/name:material.extensions.emoji.twemoji
emoji_generator: !!python/name:material.extensions.emoji.to_svg
Loading