Skip to content
This repository has been archived by the owner on Feb 15, 2022. It is now read-only.

Three strategies have been coded to the best of my ability. + Lightning Fast Threading #709

Closed
wants to merge 8 commits into from

Conversation

ghost
Copy link

@ghost ghost commented Nov 15, 2017

Trendline: Buy when trend exceeds > 1 sell when trend is < 1 (down) where 1 is no trend at all.
Standard Deviation: Buys when the mean of the last short trades exceeds the standard deviation and mean of the last long trades.
Neural: Uses neural regression learning to predict the next price. Can be resource intensive. Be careful. Based on weather prediction sliding max/minimum. Buys when the mean of the last two periods prediction price exceeds the mean of the last 3 periods actual prices. I call it sliding 2/3rds averaging prediciton signal.

Also:
I attempted a fix for zenbot's single-thread node limitation by incuding cluster npm module, the whole zenbot instance is clustered from the master instance for neural and other processor-resource intensive strategy simulation. Be careful, it can be prone to a memory leak with: cluster.setMaxListeners(0).
The cluster worker processes, if killed, should exit with:
cluster.on('exit', function(worker, code, signal) {
console.log('worker ' + worker.process.pid + ' died');


The entire code for clustering:

var cluster = require('cluster');
cluster.setMaxListeners(0)
var numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
// Fork workers.
for (var i = 0; i < numCPUs; i++) {
cluster.fork();
}
cluster.on('exit', function(worker, code, signal) {
console.log('worker ' + worker.process.pid + ' died');
});
} else {

////////////////////////zenbot.js code///////////////////////

}


Trendline: Buy when trend exceeds > 1 sell when trend is < 1 (down) where 1 is no trend at all.
Standard Deviation: Buys when the mean of the last short trades exceeds the standard deviation and mean of the last long trades.
Neural: Uses neural regression learning to predict the next price. Can be resource intensive. Be careful. Based on weather prediction sliding max/minimum. Buys when the mean of the last two periods prediction price exceeds the mean of the last 3 periods actual prices. I call it sliding 2/3rds averaging prediciton signal.

Also:
I attempted a fix for zenbot's single-thread node limitation by incuding cluster npm module, the whole zenbot instance is clustered from the master instance for neural and other processor-resource intensive strategy simulation. Be careful, it can be prone to a memory leak with: cluster.setMaxListeners(0).
The cluster worker processes, if killed, should exit with:
  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');

-----------------------------------------------------
The entire code for clustering:
-----------------------------------------------------
var cluster = require('cluster');
cluster.setMaxListeners(0)
var numCPUs = require('os').cpus().length;
if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < numCPUs; i++) {
    cluster.fork();
  }
  cluster.on('exit', function(worker, code, signal) {
    console.log('worker ' + worker.process.pid + ' died');
  });
} else {

////////////////////////zenbot.js code///////////////////////

}

-----------------------------------------------------
@ghost
Copy link
Author

ghost commented Nov 15, 2017

brb need to update package-lock.json to add node-prowl

@ghost
Copy link
Author

ghost commented Nov 15, 2017

Ok, I pushed in new package lock and node-prowl. This thing is way too fast.

"micro-request": "^666.0.10",
"mime": "^1.4.0",
"minimist": "^1.2.0",
"moment": "^2.18.1",
"mongodb": "^2.2.31",
"node-prowl": "^0.1.7",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have absolutely no clue how this was missing from my zenbot pull because I pulled literally straight from zenbot.git on carlos8f repo. Either way, I re-added it.

@sniper7kills
Copy link
Contributor

sniper7kills commented Nov 15, 2017

Just a suggestion; You may want to also update the README.md with your new strategies otherwise they may be lost after the pull. (I.E. no documentation people may think they don't exist)

Pulled and testing in paper mode

@ghost
Copy link
Author

ghost commented Nov 15, 2017

I noticed that list-strategies will output a bunch of times, but my speed sim was blazing fast. I mean FAST. If anyone has any suggestions, I'm open. Or if I need code edits please let me know. Or if this threading code can be dropped somewhere else, that would be amazing too. I'm mainly worried about the simulations, but not sure completely how the clustering code comes into place with say, mongodb and the nodejs code execution and the sim/trade commands. Dev testing please!!!!! Please comment any code edits I should make, but I think threading is absolutely needed for this type of speed.

@ghost
Copy link
Author

ghost commented Nov 15, 2017

I'm going to try and move the threading code back into the strategy since it did not function as desired.

TheRoboKitten and others added 3 commits November 14, 2017 23:15
Threading code did not work correctly at all.

Thinking of solutions. Not sure where to go with this. I need actual professional node advice.

```

      if (s.lookback[s.options.min_periods]) {
          for (let i = 0; i < s.options.min_periods; i++) { tll.push(s.lookback[i].close) }
          for (let i = 0; i < s.options.min_predict; i++) { tlp.push(s.lookback[i].close) }
          s.my_data = tll.reverse()
          var learn = function (s) {
            s.done = 0
            for(var j = 0; j < s.trains; j++) {
              if (cluster.isMaster) {
                  cluster.fork();
              }
              else {
                for (var i = 0; i < s.my_data.length - s.neural.neuralDepth; i++);
                  var data = s.my_data.slice(i, i + s.neural.neuralDepth);
                  var real_value = [s.my_data[i + s.neural.neuralDepth]];
                  var x = new convnetjs.Vol(data);
                  s.neural.trainer.train(x, real_value);
                  var predicted_values = s.neural.net.forward(x);
                  if (i === s.my_data.length - s.neural.neuralDepth) { s.done = j }
                  if (i === s.my_data.length - s.neural.neuralDepth) { calculating === 'False' }

                }
              }
            }
          }
          var predict = function(data){
              x = new convnetjs.Vol(data);
              predicted_value = s.neural.net.forward(x);
              return predicted_value.w[0];
          }
          if (s.lookback[s.options.min_periods] && calculating === 'False') {
          calculating = 'True'
          learn(s);
          }
          if (s.done === s.trains) {
            var item = tlp.reverse();
            s.prediction = predict(item)
            s.mean = math.mean(tll[0], tll[1], tll[2])
            s.meanp = math.mean(s.prediction, oldmean)
            s.sig0 = s.meanp > s.mean
            oldmean = s.prediction
          }

```

The above is why. How can I drop that into a new clustered function?
Had to roll back to node 7.x. But added napa/napajs microsoft threading npm module to package.json for future use.
Updated package-lock.json
Install goes smoothly.
Included install log in install.sh for diagnostics.
Hit:3 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu xenial InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
Get:5 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]
Hit:6 https://deb.nodesource.com/node_7.x xenial InRelease
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can add *.log to the .gitignore to ignore these kind of files

@@ -1,3199 +0,0 @@
{
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you removed the entire package-lock.json?

This pull request was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants