Skip to content

Commit

Permalink
Added addition getting started resources.
Browse files Browse the repository at this point in the history
  • Loading branch information
kenstott committed Aug 30, 2024
1 parent e3e440d commit d8f082e
Show file tree
Hide file tree
Showing 8 changed files with 745 additions and 21 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ target/
/adapters/hive/beeline.conf/
/adapters/hive/hadoop_data/

/test-connector/
200 changes: 199 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,202 @@
# NDC Calcite

This repository contains a collection of adapters built on top of Apache Calcite..
This repository contains an adapter that is metadata configurable to support approximately 40 data sources.

Approximately 15 files-based data sources, and 25 JDBC based data sources.

## Temporary Instructions - For Getting Started as a Developer with this repo.

### Clone the repo && the subrepo

This adapter is based on a forked version of Calcite (the sub-repo)

```shell
git clone --recurse-submodules https://github.com/hasura/ndc-calcite.git calcite-connectioncd calcite-connector
cd calciate-connector
git checkout main
```

Note - this is somewhat simplified - because everything is in the "main" branch. I'll let you research how to manage
the primary and sub-branch on your own!

### Build the Java project

The project will require jdk 21 and maven. You need to have those installed first.

This is the JNI for calcite. It handles the Calcite to Rust handoff.

You can build it like this.
```shell
cd calcite-rs-jni
chmod +x build.sh
./build.sh
```

This will build the Java jars that the Rust project (at the root of this mono-repo) requires.

### Build the Connector and CLI Plugin

```shell
cd ..
cargo build --bin ndc-calcite --bin ndc-calcite-cli
```

### Test the file adapter

Note - the test requires "metadata" to be added to configuration.json.
It should do it on first pass, but for now - you may to run it once to populate.
Then run it again to actually perform the test.

```shell
chmod +x test.sh
./test.sh # populate metadata
./test.sh # run the tests
```
### Build the docker image

```shell
chmod +x build-local.sh
./build-local.sh
```

### Create a supergraph

```shell
ddn supergraph init test-connector
cd test-connector
```

### Create a connector under default subgraph "app"
```shell
mkdir ./app/connector
ddn connector-link add calcite --configure-connector-token secret --configure-host http://local.hasura.dev:8081 --subgraph app/subgraph.yaml --target-env-file .env
```

### Add metadata to the connector

This script is one-and-done, you can't redo with resetting back to prior state.
You might consider, committing after prior state, to facilitate a rollback.
```shell
chmod +x ../cli.sh
../cli.sh ./app/connector/calcite 8081 secret
```

### Optional Revise Calcite Adapter

This will setup a SQLite connector. If you want to change the connector DO IT NOW. Got `app/connector/calcite/models/model.json` and revise the schema(s).
Look at the sample models for ideas, or, get more details from [Apache Calcite](https://calcite.apache.org/docs/adapter.html).

```shell
chmod +x ../cli-update-model.sh
../cli-update-model.sh
```

### Start supergraph

This is to facilitate the introspection. Introspection will not work offline
with `ddn connect-link add-all`, without the connector being in connector hub.
(That's a guess, since I can't prove it.)

```shell
HASURA_DDN_PAT=$(ddn auth print-pat) docker compose --env-file .env up --build --watch
```

### Introspect

```shell
ddn connector-link update calcite --add-all-resources --subgraph app/subgraph.yaml
```

### Build supergraph

```shell
ddn supergraph build local
```

### View in console

[Click here to launch Console View](https://console.hasura.io/local/graphql?url=http://localhost:3000)

### Execute a query

```graphql
query MyQuery {
albums(limit: 10) {
title
artist {
name
}
}
}
```

And you should see this:

```json
{
"data": {
"albums": [
{
"title": "For Those About To Rock We Salute You",
"artist": {
"name": "AC/DC"
}
},
{
"title": "Balls to the Wall",
"artist": {
"name": "Accept"
}
},
{
"title": "Restless and Wild",
"artist": {
"name": "Accept"
}
},
{
"title": "Let There Be Rock",
"artist": {
"name": "AC/DC"
}
},
{
"title": "Big Ones",
"artist": {
"name": "Aerosmith"
}
},
{
"title": "Jagged Little Pill",
"artist": {
"name": "Alanis Morissette"
}
},
{
"title": "Facelift",
"artist": {
"name": "Alice In Chains"
}
},
{
"title": "Warner 25 Anos",
"artist": {
"name": "Antônio Carlos Jobim"
}
},
{
"title": "Plays Metallica By Four Cellos",
"artist": {
"name": "Apocalyptica"
}
},
{
"title": "Audioslave",
"artist": {
"name": "Audioslave"
}
}
]
}
}
```
Loading

0 comments on commit d8f082e

Please sign in to comment.