Refactor pricenode

.editorconfig

@@ -3,7 +3,7 @@ root = true
 [*]
 indent_style = space
 indent_size = 4
-continuation_indent_size = 8
+continuation_indent_size = 4
 end_of_line = lf
 charset = utf-8
 trim_trailing_whitespace = true

.gitignore

@@ -1,6 +1,17 @@
+# Vim
 *.swp
+
+# IDEA
 .idea
 *.iml
+out
+
+# Gradle
 build
 .gradle
-out
+
+# Heroku
+.heroku/
+.jdk/
+.profile.d/
+tor/

.travis.yml

@@ -0,0 +1,2 @@
+language: java
+jdk: oraclejdk8

Procfile

@@ -1 +1 @@
-web: if [ "$HIDDEN" == true ]; then ./tor/bin/run_tor build/install/bisq-pricenode/bin/bisq-pricenode 2 5 3; else build/install/bisq-pricenode/bin/bisq-pricenode 2 5 3; fi
+web: if [ "$HIDDEN" == true ]; then ./tor/bin/run_tor java -jar -Dserver.port=$PORT build/libs/bisq-pricenode.jar 2 5 3; else java -jar -Dserver.port=$PORT build/libs/bisq-pricenode.jar 2 5 3; fi

README-HEROKU.md

@@ -4,6 +4,7 @@ Deploy on Heroku
 Run the following commands:
 
     heroku create
+    heroku buildpacks:add heroku/gradle
     heroku config:set BITCOIN_AVG_PUBKEY=[your pubkey] BITCOIN_AVG_PRIVKEY=[your privkey]
     git push heroku master
     curl https://your-app-123456.herokuapp.com/getAllMarketPrices

README.md

@@ -5,54 +5,85 @@ bisq-pricenode
 Overview
 --------
 
-The Bisq price relay node ("pricenode" for short) is a simple HTTP service that fetches data from third-party price providers and relays that data to Bisq exchange clients on request. Available prices include:
+The Bisq pricenode is a simple HTTP service that fetches, transforms and relays data from third-party price providers to Bisq exchange clients on request. Available prices include:
 
- - BTC exchange rates, available at `/getAllMarketPrices`, and
- - BTC mining fees, available at `/getFees`
+ - Bitcoin exchange rates, available at `/getAllMarketPrices`, and
+ - Bitcoin, Litecoin, Dash, and Dogecoin mining fee rates, available at `/getFees`
 
-Price relays are deployed as Tor hidden services. This is not because the location of these nodes needs to be kept secret, but rather so that Bisq exchange clients do not need to exit the Tor network in order to get price data.
+Pricenodes are deployed in production as Tor hidden services. This is not because the location of these nodes needs to be kept secret, but rather so that Bisq exchange clients do not need to exit the Tor network in order to get price data.
 
-Anyone can run a price relay, but a relay must be _discoverable_ in order for it to do any good. For exchange clients to discover your relay, its .onion address must be (a) hard coded in the Bisq exchange client's `ProviderRepository` class or (b) specified explicitly via the exchange client's `--providers` command line option.
+Anyone can run a pricenode, but it must be _discoverable_ in order for it to do any good. For exchange clients to discover your pricenode, its .onion address must be hard-coded in the Bisq exchange client's `ProvidersRepository` class. Alternatively, users can point explicitly to given pricenode (or set of pricenodes) with the exchange client's `--providers` command line option.
 
-Price relays can be run anywhere Java and Tor binaries can be run. Instructions below cover deployment on localhost and Heroku, but nothing in principle prevents these nodes from being run across a broader selection of platforms and regions. Note however that the exchange client is currently naive about selecting price relays (it does so once randomly at startup), but this could be improved in various ways to accommodate a larger number of better-distributed nodes.
+Pricenodes can be deployed anywhere Java and Tor binaries can be run. Instructions below cover deployment on localhost, and instructions [how to deploy on Heroku](README-HEROKU.md) are also available.
 
-Price relays should be cheap to run with regard to both time and money. The application itself is non-resource intensive and can be run on the low-end of most providers' paid tiers, and possibly even on some providers' free tiers.
+Pricenodes should be cheap to run with regard to both time and money. The application itself is non-resource intensive and can be run on the low-end of most providers' paid tiers.
 
-A price relay operator's main responsibilities are to ensure their node(s) are available and up-to-date. Releases are currently source-only, with the assumption that most operators will favor Git-based "push to deploy" workflows. To stay up to date with releases, operators can [subscribe to this repository's releases.atom feed](https://github.com/bisq-network/pricenode/releases.atom).
+A [pricenode operator](https://github.com/bisq-network/roles/issues/5)'s main responsibilities are to ensure their node(s) are available and up-to-date. Releases are currently source-only, with the assumption that most operators will favor Git-based "push to deploy" workflows. To stay up to date with releases, operators can [subscribe to this repository's releases.atom feed](https://github.com/bisq-network/pricenode/releases.atom) and/or get notifications in the `#pricenode` Slack channel.
 
-Operating a production price relay is a valuable service to the Bisq network. Accordingly, operators should issue BSQ compensation requests that cover their time and money costs.
+Operating a production pricenode is a valuable service to the Bisq network, and operators should issue BSQ compensation requests accordingly.
 
 
-Prerequisites
+Prerequisites for running a pricenode
 --------
 
-To run a price relay, you will need:
+To run a pricenode, you will need:
 
   - [BitcoinAverage API keys](https://bitcoinaverage.com/en/plans). Free plans are fine for local development or personal nodes; paid plans should be used for well-known production nodes.
-  - [Tor Browser](https://www.torproject.org/projects/torbrowser.html.en) to verify .onion URLs.
   - JDK 8 if you want to build and run a node locally.
   - The `tor` binary (e.g. `brew install tor`) if you want to run a hidden service locally.
 
 
-Deploy Locally
+How to deploy locally
 --------
 
-Run the following commands:
+### Configure
+
+Export the following properties as environment variables, e.g.:
+
+    $ export BITCOIN_AVG_PUBKEY=[your pubkey]
+    $ export BITCOIN_AVG_PRIVKEY=[your privkey]
+
+Or add them to your `bisq.properties` config file, e.g.:
+
+    $ echo BITCOIN_AVG_PUBKEY=[your pubkey] >> $HOME/.config/bisq.properties
+    $ echo BITCOIN_AVG_PRIVKEY=[your privkey] >> $HOME/.config/bisq.properties
+
+> TIP: Using the `bisq.properties` config file has the advantage of not needing to specify environment variables in your IDE. Running the app and running tests will "just work" regardless where and how you run them.
+
+### Build
+
+    ./gradlew assemble
+
+### Run
+
+    java -jar ./build/libs/bisq-pricenode.jar [capacity] [max-blocks] [request-interval-mins]
+
+### Test
+
+To manually test endpoints, run each of the following:
 
-    ./gradlew installDist
-    BITCOIN_AVG_PUBKEY=[your pubkey] BITCOIN_AVG_PRIVKEY=[your privkey] ./build/install/bisq-pricenode/bin/bisq-pricenode [capacity] [max-blocks] [request-interval-mins]
     curl http://localhost:8080/getAllMarketPrices
+    curl http://localhost:8080/getFees
+    curl http://localhost:8080/getParams
+    curl http://localhost:8080/getVersion
+    curl http://localhost:8080/info
+
+### Run as Tor hidden service
 
-To register the node as a Tor hidden service, run:
+With your pricenode running at localhost:8080, run:
 
     tor -f torrc
+
+Wait for the process to report that it is "100% bootstrapped", then copy your newly-generated .onion address:
+
+    export PRICENODE_ONION=$(cat build/tor-hidden-service/hostname)
 
-When the process reports that it is "100% bootstrapped", leave the process running, copy your newly-generated .onion address from `build/tor-hidden-service/hostname` and then open http://$YOUR_ONION/getAllMarketPrices in your Tor Browser. You should see the same response as produced in the `curl` request above.
+Test the endpoints of your hidden service via curl with the --socks5-proxy option:
 
-> NOTE: It may take a few minutes for your new .onion address to get registered and become resolvable. Registration of Tor hidden service descriptors can take some time.
+    curl --socks5-hostname 127.0.0.1:9050 http://$PRICENODE_ONION/getAllMarketPrices
 
 
-Deploy Elsewhere
+How to deploy elsewhere
 --------
 
  - [README-HEROKU.md](README-HEROKU.md)

TODO.md

@@ -0,0 +1,20 @@
+# Refactorings
+
+The list of stuff remaining to complete the PR at https://github.com/bisq-network/pricenode/pull/7
+
+ - Document provider implementations w/ links to API docs, etc
+ - Add integration tests
+ - Document / discuss how operators should (ideally) operate their pricenodes on a push-to-deploy model, e.g. how it's done on Heroku
+
+## Non-refactorings
+
+Most or all of these will become individual issues / PRs. Just capturing them here for convenience now. Not all may make sense.
+
+ - Deprecate existing get* endpoints (e.g. /getAllMarketPrices) in favor of '/exchange-rates', '/fee-estimate;
+ - Eliminate dependency on bisq-core (only real need now is CurrencyUtil for list of supported coins)
+ - Remove command line args for fee estimation params; hard-code these values and update them via commits, not via one-off changes by each operator
+ - Remove 'getParams' in favor of Boot actuator endpoint
+ - Update bisq-network/exchange to refer to 'provider' as 'pricenode'
+ - Invert the dependency arrangement. Move 'ProviderRepository' et al from bisq-network/exchange here into
+   bisq-network/pricenode and have bisq-network/exchange depend on it as a client lib
+ - Save bandwidth and be idiomatic by not pretty-printing json returned from /getAllMarketPrices et al

build.gradle

@@ -1,29 +1,33 @@
 plugins {
     id "java"
-    id "application"
+    id "org.springframework.boot" version "1.5.10.RELEASE"
 }
 
 sourceCompatibility = 1.8
 targetCompatibility = 1.8
 
-mainClassName = "bisq.pricenode.Main"
-version = "0.7.0"
+version = file("src/main/resources/version.txt").text
 
 jar.manifest.attributes(
-        "Implementation-Title": rootProject.name,
-        "Implementation-Version": version)
+    "Implementation-Title": rootProject.name,
+    "Implementation-Version": version)
+
+jar.archiveName "${rootProject.name}.jar"
 
 repositories {
     jcenter()
     maven { url "https://jitpack.io" }
-    maven { url "https://raw.githubusercontent.com/JesusMcCloud/tor-binary/testrelease/release/" }
 }
 
 dependencies {
-    compile("com.github.bisq-network.bisq-exchange:core:v0.6.4")
-    compile("com.sparkjava:spark-core:2.5.2")
+    compile("com.github.bisq-network.bisq-exchange:common:v0.6.5")
+    compile("org.knowm.xchange:xchange-bitcoinaverage:4.3.3")
+    compile("org.knowm.xchange:xchange-coinmarketcap:4.3.3")
+    compile("org.knowm.xchange:xchange-poloniex:4.3.3")
+    compile("org.springframework.boot:spring-boot-starter-web:1.5.10.RELEASE")
+    compile("org.springframework.boot:spring-boot-starter-actuator")
 }
 
 task stage {
-    dependsOn installDist
+    dependsOn assemble
 }

src/main/java/bisq/price/Main.java

@@ -0,0 +1,51 @@
+/*
+ * This file is part of Bisq.
+ *
+ * Bisq is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU Affero General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * Bisq is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ * License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with Bisq. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package bisq.price;
+
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+import org.springframework.boot.builder.SpringApplicationBuilder;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.UncheckedIOException;
+
+import java.util.Properties;
+
+@SpringBootApplication
+public class Main {
+
+    public static void main(String[] args) {
+        new SpringApplicationBuilder(Main.class)
+            .properties(bisqProperties())
+            .run(args);
+    }
+
+    private static Properties bisqProperties() {
+        Properties props = new Properties();
+        File propsFile = new File(System.getenv("HOME"), ".config/bisq.properties");
+        if (propsFile.exists()) {
+            try {
+                props.load(new FileInputStream(propsFile));
+            } catch (IOException ex) {
+                throw new UncheckedIOException(ex);
+            }
+        }
+        return props;
+    }
+}

src/main/java/bisq/price/PriceController.java

@@ -0,0 +1,35 @@
+/*
+ * This file is part of Bisq.
+ *
+ * Bisq is free software: you can redistribute it and/or modify it
+ * under the terms of the GNU Affero General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or (at
+ * your option) any later version.
+ *
+ * Bisq is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+ * License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with Bisq. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package bisq.price;
+
+import org.springframework.web.bind.annotation.ModelAttribute;
+
+import javax.servlet.http.HttpServletRequest;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public abstract class PriceController {
+
+    protected final Logger log = LoggerFactory.getLogger(this.getClass());
+
+    @ModelAttribute
+    public void logRequest(HttpServletRequest request) {
+        log.info("Incoming {} request from: {}", request.getServletPath(), request.getHeader("User-Agent"));
+    }
+}