JS Port/Maintenance/Feed Sources

[clockworkgr]

Seeing as this is not heavily maintained and my python skills are limited, I am porting the entire thing over to node/JS (about 80% done).

I am using this issue to notify users that it will be published on github soon and also to gather feedback & feature requests to be implemented on my port as well as new Feed sources.

I would also appreciate if people could notify me of sources that no longer work (e.g. I believe AEX api no longer works)

[Zapata]

Python is not my main/favorite programming language but I managed to review all the code, test all the sources and formulas, found and reported some issues and fixed most of them in my repo.

Current status on sources is:

• Should be removed: BTC38, BTCChina, ChBTC?, BTer, Yunbi (Remove closed exchanges. #40), Yahoo (Remove Yahoo Finance API. #41)
• Should be rewritten: BitcoinVenezuela ( Fix source Bitcoin Venezuela. #42), Big.One (Fix Big.ONE source: to be implemented. #43), BitcoinAverage (Fix Bitcoin Average source: upgrade API. #44), Fixer (Fix Fixer source: upgrade to new API. #49), Huobi (Fix Huobi source: upgrade API #51)
• Works with bugs: Coincap (ALTCAP.X) (Fix ALTCAP.X computation in Coincap source. #45), Coinmarketcap (Coinmarketcap: If multiple quotes are provided only the last one is returned. #46, Coinmarketcap: ALTCAP and ALTCAP.X are computed but ignored. #47, Coinmarketcap: Migrate to API v2. #48), Graphene (Fix Graphene source: If multiple quotes are provided only the last one is returned. #50), Quandl (Quandl: Quote inconsistency between Quandl and QuandlPlain. #52)

On my repo, I have fixed most of them except: Big.ONE, Bitcoin Average, Huobi. I could do that in the coming weeks if required.
I have added unit tests to quickly check all the sources and some documentation.

It's good to have alternative implementation in other languages as currently there is only Python implementations (alt/bsprice or wackou/bts_tools).
But none of the current implementations are well maintained, and with good quality (clean code, tested, easy debug, transparent price computation, well documented).
I suggest we focus our efforts on one implementation (existing or new).

[clockworkgr]

well..in case you're interested, my code is here : https://github.com/clockworkgr/bitshares-pricefeed-js

Pretty much feature complete...needs clean up which i plan to do in the next few days...

should be easy to expand...will document the process

[Zapata]

Yes I've seen your code.

I don't know what you mean by feature complete, but for me it means:

• maintenable (clean code, well documented, unit test, ...)
• Handle all the MPA that Bitshares handle:
  -- Crypto (BTC, ..)
  -- FIAT (USD, CNY, EUR, ..)
  -- Algoritmic (HERTZ, HERO, ALTCAP, ...)
  -- Stocks (AMAZONCOM, FACEBOOKCOM, ...)
  -- ETFs (Bit20, Twentix, S&P, ...)
• Good traceability: it should be possible to know how each price was computed, even days after their publication (which sources was used, what price they provided, etc...)
• Good checks/alerting/error workflows to avoid publication of wrong prices when something went wrong (like bad data provided by the sources as it's commonly the case with manipulated centralized exchanged), and detect issues in other feeds.

I want to emphasis the last points, I used to work for price publication services of financial institutions and those are the kind of expectations they have. If we want that one day institutional business use MPA, we need professional publication tools.

[clockworkgr]

clean code and documented is part of the clean up to be done

Unit testing i make no promises as I hate writing them :)

Traceability.. interesting. I hadn't thought about it but I can think of a few ways to store that data nicely.

Again...should be all there but clean up will help in that respect.

Thanks for your input... It really helps make this better and already given me some food for thought

[clockworkgr]

@Zapata With regards to traceability, how does this sound? On every run, create a JSON object containing a timestamp, an array of the symbols actually published, the working price_feed object containing all the source data retrieved and the config object. serialize and save (could also compress first)

[Zapata]

Perfect, you've got it!