|
1 |
| -# Vulcan SQL |
| 1 | +<p align="center"> |
| 2 | + <img src="https://i.imgur.com/GJsuthW.png" width="600" > |
| 3 | +</p> |
| 4 | + |
| 5 | +<p align="center"> |
| 6 | + <a aria-label="Canner" href="https://cannerdata.com/"> |
| 7 | + <img src="https://img.shields.io/badge/%F0%9F%A7%A1-Made%20by%20Canner-orange?style=for-the-badge"> |
| 8 | + </a> |
| 9 | + <a aria-label="NPM version" href="https://www.npmjs.com/package/@vulcan-sql/core"> |
| 10 | + <img alt="" src="https://img.shields.io/npm/v/@vulcan-sql/core?color=orange&style=for-the-badge"> |
| 11 | + </a> |
| 12 | + <a aria-label="License" href="https://github.com/Canner/vulcan/blob/develop/LICENSE"> |
| 13 | + <img alt="" src="https://img.shields.io/github/license/Canner/vulcan?color=orange&style=for-the-badge"> |
| 14 | + </a> |
| 15 | + <a aria-label="Join the community on GitHub" href="https://join.slack.com/t/vulcan-bhi6765/shared_invite/zt-1dzixpy38-pKrcewZ6eM3wSqAs6~is8Q"> |
| 16 | + <img alt="" src="https://img.shields.io/badge/-JOIN%20THE%20COMMUNITY-orange?style=for-the-badge&logo=Slack&labelColor=black&logoWidth=20"> |
| 17 | + </a> |
| 18 | +</p> |
| 19 | + |
| 20 | +> 🚀 Release in Sep 2022. |
| 21 | +
|
| 22 | +## Why Vulcan |
| 23 | +> ⚡️ Data analyst / analytical engineers’ time should focus on important matters like data transformation and communicating with data consumers on high level. |
| 24 | +
|
| 25 | +<p align="center"> |
| 26 | + <img src="https://i.imgur.com/OTFP6Qh.jpg" width="800" > |
| 27 | +</p> |
| 28 | + |
| 29 | +Data analysts and analytical engineers often take lots of time helping data consumers. Backend engineers who build internal dashboards will ask about data catalog, documentations and if there are any APIs they can directly use. Business users will ask what data they can use to achieve their goals, and how to get data to their spreadsheet or excel. |
| 30 | + |
| 31 | +<p align="center"> |
| 32 | + <img src="https://i.imgur.com/dn5kzXC.png" width="800" > |
| 33 | +</p> |
| 34 | + |
| 35 | +With Vulcan, we prepare what data consumers need for you. Imagine you can **unify the data access by building APIs instantly with just SQL. Authorization, validation, pagination …etc are all included automatically.** |
| 36 | + |
| 37 | +Vulcan also builds documentations and a self-served catalog, so **data consumers can understand the data and get data from the tools they’re using all by themselves without using any SQL**. |
| 38 | + |
| 39 | +## Features |
| 40 | +- Build API instantly with just SQL. |
| 41 | +- Access control & authorization in SQL. |
| 42 | +- API best practices included (validation, caching, pagination, sorting …etc). |
| 43 | +- API documentation is automatically built. |
| 44 | +- Self-served catalog for data consumers. A step-by-step guide to get data from Excel / Google spreadsheet, Zapier, Retools …etc. |
| 45 | + |
| 46 | +## How Vulcan works? |
| 47 | + |
| 48 | +### Step 1: Instant API with just SQL. |
| 49 | +<p align="center"> |
| 50 | + <img src="https://i.imgur.com/2PMrlJC.png" width="600" > |
| 51 | +</p> |
| 52 | + |
| 53 | +Building API with just SQL. No complex web framework and business logic. |
| 54 | + |
| 55 | +**Example: passing parameters from url** |
| 56 | + |
| 57 | +```sql |
| 58 | +select * from public.users where id = '{{ params.userId }}' |
| 59 | +``` |
| 60 | + |
| 61 | +You can build an API endpoint `users` with `userId` as input. |
| 62 | + |
| 63 | +API users will be able to get data like |
| 64 | + |
| 65 | +```js |
| 66 | +GET /users?userId=1 |
| 67 | + |
| 68 | +Response |
| 69 | +[{ |
| 70 | + "name": "wwwy3y3", |
| 71 | + "age": 30 |
| 72 | +}] |
| 73 | +``` |
| 74 | +
|
| 75 | +#### **Other Examples:** |
| 76 | +
|
| 77 | +<details> |
| 78 | + <summary>1. Error Handling</summary> |
| 79 | +
|
| 80 | + If you want to throw errors based on data, for example, run a query first, if no data return, throw `404 not found`. |
| 81 | + |
| 82 | + ```sql |
| 83 | + {% req user %} |
| 84 | + select * from public.users where userName = '{{ parames.userName }}'; |
| 85 | + {% endreq %} |
| 86 | + |
| 87 | + {% if user.count().value() == 0 %} |
| 88 | + {% error "user not found" %} |
| 89 | + {% endif %} |
| 90 | + |
| 91 | + select * from public.groups where userId = '{{ user.value()[0].id }}'; |
| 92 | + ``` |
| 93 | +</details> |
| 94 | +
|
| 95 | +<details> |
| 96 | + <summary>2. Authorization</summary> |
| 97 | +
|
| 98 | + You can pass in user attributes to SQL to control the access. |
| 99 | + |
| 100 | + ```sql |
| 101 | + select |
| 102 | + --- masking address if query user is not admin |
| 103 | + {% if context.user == 'ADMIN' %} |
| 104 | + {% "address" %} |
| 105 | + {% elif %} |
| 106 | + {% "masking(address)" %} |
| 107 | + {% endif %}, |
| 108 | + |
| 109 | + orderId, |
| 110 | + amount |
| 111 | + from orders |
| 112 | + |
| 113 | + --- limit the data to the store user belongs to. |
| 114 | + where store = {{ context.user.store }} |
| 115 | + ``` |
| 116 | +</details> |
| 117 | +
|
| 118 | +<details> |
| 119 | + <summary>3. Validation</summary> |
| 120 | +
|
| 121 | + You can add a number validator on `userId` input on your API with schema. |
| 122 | +
|
| 123 | + - SQL |
| 124 | + ```sql |
| 125 | + select * from public.users where id = '{{ params.userId }}' |
| 126 | + ``` |
| 127 | + |
| 128 | + - Schema |
| 129 | + ```yaml |
| 130 | + parameters: |
| 131 | + userId: |
| 132 | + in: query |
| 133 | + valudators: |
| 134 | + - name: 'number' |
| 135 | + ``` |
| 136 | +</details> |
| 137 | + |
| 138 | + |
| 139 | +### Step 2: Build self-served documentation and catalog |
| 140 | + |
| 141 | +Vulcan will automatically build documentation and catalog. |
| 142 | + |
| 143 | +- **Catalog**: Vulcan will build a catalog page for data consumers. This page will consist of more clear information on data that is exposed as APIs. Description, Column information are all included. |
| 144 | + |
| 145 | +<p align="center"> |
| 146 | + <img src="https://i.imgur.com/qz6swW2.png" width="800" > |
| 147 | +</p> |
| 148 | + |
| 149 | +<p align="center"> |
| 150 | + <img src="https://i.imgur.com/YZFczO3.png" width="800" > |
| 151 | +</p> |
| 152 | + |
| 153 | +- **API Documentation**: Vulcan will build a swagger page for backend engineers. |
| 154 | + |
| 155 | +<p align="center"> |
| 156 | + <img src="https://i.imgur.com/oH9UEoD.png" width="800" > |
| 157 | +</p> |
| 158 | + |
| 159 | +### Step 3: Connect from framework & applications |
| 160 | + |
| 161 | +On catalog page, you can preview data here and connect from your own framework and applications. |
| 162 | + |
| 163 | +- You can `Copy API URL` to use it at frontend / backend. |
| 164 | +- You can download the selected data as CSV or JSON. |
| 165 | + |
| 166 | +<p align="center"> |
| 167 | + <img src="https://i.imgur.com/YZFczO3.png" width="800" > |
| 168 | +</p> |
| 169 | + |
| 170 | +- You can follow the steps to connect from Excel / Google Spreadsheet / Zapier / Retools. |
| 171 | + |
| 172 | +<p align="center"> |
| 173 | + <img src="https://i.imgur.com/zOEdRYT.png" width="800" > |
| 174 | +</p> |
| 175 | + |
| 176 | +## Installation |
| 177 | + |
| 178 | +- npm / yarn |
| 179 | + ```bash |
| 180 | + npm install @vulcan-sql/cli |
| 181 | + yarn add @vulcan-sql/cli |
| 182 | + ``` |
| 183 | + |
| 184 | +- brew |
| 185 | + ```sql |
| 186 | + brew install vulcan |
| 187 | + ``` |
| 188 | + |
| 189 | + |
| 190 | +## Quickstart |
| 191 | + |
| 192 | +1. Initialize a Vulcan project |
| 193 | + ```bash |
| 194 | + vulcan init --name my-first-vulcan-project && cd my-first-vulcan-project |
| 195 | + ``` |
| 196 | + |
| 197 | +2. Start Vulcan server. |
| 198 | + ```bash |
| 199 | + vulcan start --watch |
| 200 | + ``` |
| 201 | + |
| 202 | +3. Visit API & Catalog |
| 203 | + ```bash |
| 204 | + Vulcan started... |
| 205 | + Visit API document at http://localhost:8080/doc |
| 206 | + Visit Catalog at http://localhost:8080/catalog |
| 207 | + ``` |
2 | 208 |
|
3 |
| -This project was generated using [Nx](https://nx.dev). |
| 209 | +## Demo Screenshot |
| 210 | +<p align="center"> |
| 211 | + <img src="https://i.imgur.com/j4jcYj1.png" width="800" > |
| 212 | +</p> |
4 | 213 |
|
5 |
| -<p style="text-align: center;"><img src="https://raw.githubusercontent.com/nrwl/nx/master/images/nx-logo.png" width="450"></p> |
| 214 | +> 🔑 **Login Page**: data consumers will be asked to authenticate themselves. |
6 | 215 |
|
7 |
| -🔎 **Smart, Fast and Extensible Build System** |
| 216 | +<p align="center"> |
| 217 | + <img src="https://i.imgur.com/0VmXMpl.png" width="800" > |
| 218 | +</p> |
8 | 219 |
|
9 |
| -## Adding capabilities to your workspace |
| 220 | +> 📖 **Catalog**: After logged-in, data consumers can see what endpoints are available to be used. |
10 | 221 |
|
11 |
| -Nx supports many plugins which add capabilities for developing different types of applications and different tools. |
| 222 | +<p align="center"> |
| 223 | + <img src="https://i.imgur.com/YZFczO3.png" width="800" > |
| 224 | +</p> |
12 | 225 |
|
13 |
| -These capabilities include generating applications, libraries, etc as well as the devtools to test, and build projects as well. |
| 226 | +> ✅ **Endpoint**: data consumers can view the detail of one endpoint and preview the data. |
14 | 227 |
|
15 |
| -Below are our core plugins: |
| 228 | +<p align="center"> |
| 229 | + <img src="https://i.imgur.com/zOEdRYT.png" width="800" > |
| 230 | +</p> |
16 | 231 |
|
17 |
| -- [React](https://reactjs.org) |
18 |
| - - `npm install --save-dev @nrwl/react` |
19 |
| -- Web (no framework frontends) |
20 |
| - - `npm install --save-dev @nrwl/web` |
21 |
| -- [Angular](https://angular.io) |
22 |
| - - `npm install --save-dev @nrwl/angular` |
23 |
| -- [Nest](https://nestjs.com) |
24 |
| - - `npm install --save-dev @nrwl/nest` |
25 |
| -- [Express](https://expressjs.com) |
26 |
| - - `npm install --save-dev @nrwl/express` |
27 |
| -- [Node](https://nodejs.org) |
28 |
| - - `npm install --save-dev @nrwl/node` |
| 232 | +> 🔌 **Connect**: data consumers will be able to follow the guide and connect from their applications. |
29 | 233 |
|
30 |
| -There are also many [community plugins](https://nx.dev/community) you could add. |
| 234 | +## Community |
| 235 | +* Welcome to our [Slack](https://join.slack.com/t/vulcan-bhi6765/shared_invite/zt-1dzixpy38-pKrcewZ6eM3wSqAs6~is8Q) to give us feedbacks! |
| 236 | +* If any issues, please visit [Github Issues](https://github.com/Canner/vulcan/issues) |
31 | 237 |
|
32 |
| -## Generate an application |
33 |
| - |
34 |
| -Run `nx g @nrwl/react:app my-app` to generate an application. |
35 |
| - |
36 |
| -> You can use any of the plugins above to generate applications as well. |
37 |
| -
|
38 |
| -When using Nx, you can create multiple applications and libraries in the same workspace. |
39 |
| - |
40 |
| -## Generate a library |
41 |
| - |
42 |
| -Run `nx g @nrwl/react:lib my-lib` to generate a library. |
43 |
| - |
44 |
| -> You can also use any of the plugins above to generate libraries as well. |
45 |
| -
|
46 |
| -Libraries are shareable across libraries and applications. They can be imported from `@vulcan-sql/mylib`. |
47 |
| - |
48 |
| -## Development server |
49 |
| - |
50 |
| -Run `nx serve my-app` for a dev server. Navigate to http://localhost:4200/. The app will automatically reload if you change any of the source files. |
51 |
| - |
52 |
| -## Code scaffolding |
53 |
| - |
54 |
| -Run `nx g @nrwl/react:component my-component --project=my-app` to generate a new component. |
55 |
| - |
56 |
| -## Build |
57 |
| - |
58 |
| -Run `nx build my-app` to build the project. The build artifacts will be stored in the `dist/` directory. Use the `--prod` flag for a production build. |
59 |
| - |
60 |
| -## Running unit tests |
61 |
| - |
62 |
| -Run `nx test my-app` to execute the unit tests via [Jest](https://jestjs.io). |
63 |
| - |
64 |
| -Run `nx affected:test` to execute the unit tests affected by a change. |
65 |
| - |
66 |
| -## Running end-to-end tests |
67 |
| - |
68 |
| -Run `nx e2e my-app` to execute the end-to-end tests via [Cypress](https://www.cypress.io). |
69 |
| - |
70 |
| -Run `nx affected:e2e` to execute the end-to-end tests affected by a change. |
71 |
| - |
72 |
| -## Understand your workspace |
73 |
| - |
74 |
| -Run `nx graph` to see a diagram of the dependencies of your projects. |
75 |
| - |
76 |
| -## Further help |
77 |
| - |
78 |
| -Visit the [Nx Documentation](https://nx.dev) to learn more. |
79 |
| - |
80 |
| -## ☁ Nx Cloud |
81 |
| - |
82 |
| -### Distributed Computation Caching & Distributed Task Execution |
83 |
| - |
84 |
| -<p style="text-align: center;"><img src="https://raw.githubusercontent.com/nrwl/nx/master/images/nx-cloud-card.png"></p> |
85 |
| - |
86 |
| -Nx Cloud pairs with Nx in order to enable you to build and test code more rapidly, by up to 10 times. Even teams that are new to Nx can connect to Nx Cloud and start saving time instantly. |
87 |
| - |
88 |
| -Teams using Nx gain the advantage of building full-stack applications with their preferred framework alongside Nx’s advanced code generation and project dependency graph, plus a unified experience for both frontend and backend developers. |
89 |
| - |
90 |
| -Visit [Nx Cloud](https://nx.app/) to learn more. |
|
0 commit comments