Skip to content

Commit

Permalink
put into asset folder for better file mgmt
Browse files Browse the repository at this point in the history
  • Loading branch information
kanekoshoyu committed Aug 18, 2024
1 parent 98e16d0 commit b0aee2e
Show file tree
Hide file tree
Showing 6 changed files with 58 additions and 29 deletions.
85 changes: 57 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,25 @@
# exchange_yaml: machine readable crypto exchange API doc in OpenAPI/AsyncAPI YAML
# exchange_yaml: list of crypto exchange API doc in OpenAPI/AsyncAPI
[![doc](https://img.shields.io/badge/doc-rapidoc-blue)](https://example.com)
[![license](https://img.shields.io/github/license/kanekoshoyu/kucoin_arbitrage)](https://github.com/kanekoshoyu/kucoin_arbitrage/blob/master/LICENSE)
[![ci](https://img.shields.io/github/actions/workflow/status/kanekoshoyu/kucoin_arbitrage/rust.yml)](https://github.com/kanekoshoyu/kucoin_arbitrage/actions)
[![issues](https://img.shields.io/github/issues/kanekoshoyu/kucoin_arbitrage)](https://github.com/kanekoshoyu/kucoin_arbitrage/issues)
[![discussions](https://img.shields.io/github/discussions/kanekoshoyu/kucoin_arbitrage)](https://github.com/kanekoshoyu/kucoin_arbitrage/discussions)
[![discord](https://img.shields.io/discord/1153997271294283827)](https://discord.gg/q3j5MYdwnm)

[API doc here](https://kanekoshoyu.github.io/exchange_yaml)
## problem with other cross-exchange libraries/frameworks
Most of the trading bots require data and execution at different exchanges. There are trading libraries that provides abstraction over multiple exchanges.
- [ccxt](https://github.com/ccxt/ccxt)
- [hummingbot](https://github.com/hummingbot/hummingbot)
- [barter_rs](https://github.com/barter-rs/barter-rs)

## problem
Most of the trading bots require data and execution at different exchanges. There are trading libraries like [CCXT](https://github.com/ccxt/ccxt) and [barter](https://github.com/barter-rs/barter-rs) that provides abstraction over multiple exchanges. However, they often miss support in either exchanges or languages.
- high effort: there are way too many crypto exchanges and programming languages. Say there are N crypto exchanges, and L programming language, and the effort to convert those written API doc into exchange library is B, then the overall effort is N * L * B.
- versioning: crypto exchange usually have lots of API updates and takes lots of manual testing, and it comes with inconsistent API doc as well
- too flavorul: these cross-exchange libraries often goes with their bigger project/workspace, making it too restrictive to fulfill your actual needs.
However, they often miss support in either exchanges or languages.
- high marginal effort: there are way too many crypto exchanges and programming languages. Say there are N crypto exchanges, and L programming language, and the effort to convert those written API doc into exchange library is B, then the overall effort is N * L * B.
- poor versioning: crypto exchange usually have lots of API updates and takes lots of manual testing, and it comes with inconsistent API doc as well
- highly opinionated models: these cross-exchange libraries often goes with their bigger project/workspace, and unfortunately there is no thoughts in making them generic, so often you cannot use their models directly to fulfill your actual needs.

## solution
Ultimately I want to streamline this process of integrating multi-exchange API. I propose a new approach to this problem with the help of crowdsourcing and opensourcing, and try to generate a bunch of generic code ready for your use.
1. gather both OpenAPI YAML and AsyncAPI YAML
I want to streamline the process of integrating multi-exchange API with a new approach to this problem. The idea is to gather a list of OpenAPI/AsyncAPI from both official and unofficial sources under a few guidelines, and try to generate a bunch of generic code ready for your use.
1. gather both OpenAPI and AsyncAPI YAML
(effort: N * S, S is the smaller effort of gathering / testing YAML from written API)
- OpenAPI: define REST API
- AsyncAPI: define event-driven API (websocket)
Expand All @@ -30,40 +39,60 @@ To kick off, I will gather a bunch of AsyncAPI YAML here. In the near future I w
| KuCoin | planned | planned |
| ByBit | planned | planned |

## guidelines
1. OpenAPI: YAML, v3+
2. AsyncAPI: YAML, v2 (codegen does not work well with v3 apparently)
3. official codegen support: rust, python
4. unofficial support: typescript, csharp, golang, java, javascript, dart,kotlin, php, cplusplus, scala
5. place YAML in follow the naming convension of `{exchange}_{asyncapi/openapi}.yaml`
6. codegen scripting in python

## codegen commands
below are the commands
### OpenAPI
wip
#### initial set up
install OpenAPI CLI
```
npm install -g @openapitools/openapi-generator-cli
```
#### rust model
```
openapi-generator-cli generate -i path/to/your/openapi.yaml -g <language> -o path/to/output/directory
```
#### rust model
```
openapi-generator-cli generate -i path/to/your/openapi.yaml -g <language> -o path/to/output/directory
```


### AsyncAPI
install async api code generator
#### initial set up

install AsyncAPI CLI
```
npm install -g @asyncapi/generator
```
#### rust model
```
asyncapi generate models rust example_2.yml -o output/example_rust_model
asyncapi generate models rust example_asyncapi.yml -o output/example_rust_model
```
#### python model
```
asyncapi generate models python example_2.yml -o output/example_python_model
asyncapi generate models python example_asyncapi.yml -o output/example_python_model
```
#### python paho
```
asyncapi generate fromTemplate example_2.yml @asyncapi/python-paho-template -o output/example_python_paho
```
#### python sanic (failing)
```
asyncapi generate fromTemplate example_2.yml @asyncapi/python-sanic-template -o output/example_python_sanic
asyncapi generate fromTemplate example_asyncapi.yml @asyncapi/python-paho-template -o output/example_python_paho
```

## note
- apparently the these templates only work on AsyncAPI v2, not v3 (kraken_3.yml does not work)
- model gen supports: typescript, csharp, golang, java, javascript, dart, python, rust, kotlin, php, cplusplus, scala
## todo
- [x] gather 3+ exchanges in both OpenAPI
- [ ] gather 3+ exchanges in both AsyncAPI
- [ ] verify if models can be generated on python/rust, and publish it in
- [ ] verify if traits could be implemented on top of the generated model
- [ ] package model with its version

## notes
- the `ag` command seems to be deprecated and cannot generate code properly
- you can install `asyncapi-preview` extension on vs code for preview
- follow the naming convension of `{exchange}_{asyncapi/openapi}.yaml` for easier browsing and scripting.

## todo
- gather 3+ exchanges in both OpenAPI/AsyncAPI.
- verify if the binance models can be generated on python/rust
- verify if traits could be implemented on top of the generated model
- package model with its version
- AsyncAPI `python-sanic-template` was failing
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@

exchanges.forEach(exchange => {
const option = document.createElement('option');
option.value = `https://raw.githubusercontent.com/kanekoshoyu/exchange_yaml/main/${exchange}_openapi.yaml`;
option.value = `https://raw.githubusercontent.com/kanekoshoyu/exchange_yaml/main/asset/${exchange}_openapi.yaml`;
option.text = exchange.charAt(0).toUpperCase() + exchange.slice(1);
dropdown.add(option);
});
Expand Down

0 comments on commit b0aee2e

Please sign in to comment.