Update README.md

Replace juancazala.com with caza.la
version bump for CDNJS - no actual changes
2016-09-20 14:16:38 +03:00 · 2016-09-16 19:42:46 -03:00 · 2016-09-13 12:14:35 +03:00 · 2016-09-12 01:36:48 +03:00 · 2016-08-19 18:17:12 -03:00 · 2016-08-19 22:05:07 +02:00
@@ -14,3 +14,7 @@ node_modules
 # Demo.
 demo.js

+# Degub
+debug.html
+
+.settings
@@ -1,3 +1,9 @@
 language: node_js
+ script: "npm run test:travis"
 node_js:
-   - "0.10"
+    # always latest release
+    - "node"
+    # previous releases
+    - "6"
+    - "5"
+    - "4"
@@ -1,6 +1,6 @@
 The MIT License (MIT)

-Copyright (c) 2014 Juan Cazala (juancazala.com)
+Copyright (c) 2016 Juan Cazala - juancazala.com

 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -9,14 +9,38 @@ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 copies of the Software, and to permit persons to whom the Software is
 furnished to do so, subject to the following conditions:

-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.

 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE

+
+
+********************************************************************************************
+                                  SYNAPTIC (v1.0.8)
+********************************************************************************************
+
+Synaptic is a javascript neural network library for node.js and the browser, its generalized
+algorithm is architecture-free, so you can build and train basically any type of first order
+or even second order neural network architectures.
+
+http://en.wikipedia.org/wiki/Recurrent_neural_network#Second_Order_Recurrent_Neural_Network
+
+The library includes a few built-in architectures like multilayer perceptrons, multilayer
+long-short term memory networks (LSTM) or liquid state machines, and a trainer capable of
+training any given network, and includes built-in training tasks/tests like solving an XOR,
+passing a Distracted Sequence Recall test or an Embeded Reber Grammar test.
+
+The algorithm implemented by this library has been taken from Derek D. Monner's paper:
+
+
+A generalized LSTM-like training algorithm for second-order recurrent neural networks
+http://www.overcomplete.net/papers/nn2012.pdf
+
+There are references to the equations in that paper commented through the source code.
@@ -1,6 +1,8 @@
-Synaptic [![Build Status](https://travis-ci.org/cazala/synaptic.svg?branch=master)](https://travis-ci.org/cazala/synaptic)
+Synaptic [![Build Status](https://travis-ci.org/cazala/synaptic.svg?branch=master)](https://travis-ci.org/cazala/synaptic) [![Join the chat at https://synapticjs.slack.com](https://synaptic-slack-ugiqacqvmd.now.sh/badge.svg)](https://synaptic-slack-ugiqacqvmd.now.sh/)
 ========

+## Important: [Synaptic 2.x](https://github.com/cazala/synaptic/issues/140) is in stage of discussion now! Feel free to participate
+
 Synaptic is a javascript neural network library for **node.js** and the **browser**, its generalized algorithm is architecture-free, so you can build and train basically any type of first order or even [second order neural network](http://en.wikipedia.org/wiki/Recurrent_neural_network#Second_Order_Recurrent_Neural_Network) architectures.

 This library includes a few built-in architectures like [multilayer perceptrons](http://en.wikipedia.org/wiki/Multilayer_perceptron), [multilayer long-short term memory](http://en.wikipedia.org/wiki/Long_short_term_memory) networks (LSTM), [liquid state machines](http://en.wikipedia.org/wiki/Liquid_state_machine) or [Hopfield](http://en.wikipedia.org/wiki/Hopfield_network) networks, and a trainer capable of training any given network, which includes built-in training tasks/tests like solving an XOR, completing a Distracted Sequence Recall task or an [Embedded Reber Grammar](http://www.willamette.edu/~gorr/classes/cs449/reber.html) test, so you can easily test and compare the performance of different architectures.
@@ -17,14 +19,19 @@ There are references to the equations in that paper commented through the source

 If you have no prior knowledge about Neural Networks, you should start by [reading this guide](https://github.com/cazala/synaptic/wiki/Neural-Networks-101).

+
+If you want a practical example on how to feed data to a neural network, then take a look at [this article](https://github.com/cazala/synaptic/wiki/Normalization-101).
+
+You may also want to take a look at [this article](http://blog.webkid.io/neural-networks-in-javascript/).
+
 ####Demos

- [Solve an XOR](http://synaptic.juancazala.com/#/xor)
- [Discrete Sequence Recall Task](http://synaptic.juancazala.com/#/dsr)
- [Learn Image Filters](http://synaptic.juancazala.com/#/image-filters)
- [Paint an Image](http://synaptic.juancazala.com/#/paint-an-image)
- [Self Organizing Map](http://synaptic.juancazala.com/#/self-organizing-map)
- [Read from Wikipedia](http://synaptic.juancazala.com/#/wikipedia)
+- [Solve an XOR](http://caza.la/synaptic/#/xor)
+- [Discrete Sequence Recall Task](http://caza.la/synaptic/#/dsr)
+- [Learn Image Filters](http://caza.la/synaptic/#/image-filters)
+- [Paint an Image](http://caza.la/synaptic/#/paint-an-image)
+- [Self Organizing Map](http://caza.la/synaptic/#/self-organizing-map)
+- [Read from Wikipedia](http://caza.la/synaptic/#/wikipedia)

 The source code of these demos can be found in [this branch](https://github.com/cazala/synaptic/tree/gh-pages/scripts).

@@ -36,12 +43,17 @@ The source code of these demos can be found in [this branch](https://github.com/
 - [Trainer](https://github.com/cazala/synaptic/wiki/Trainer/)
 - [Architect](https://github.com/cazala/synaptic/wiki/Architect/)

+To try out the examples, checkout the [gh-pages](https://github.com/cazala/synaptic/tree/gh-pages) branch.
+
+`git checkout gh-pages`
+

 ##Overview

 ###Installation

 #####In node
+
 You can install synaptic with [npm](http://npmjs.org):

 ```cmd
@@ -49,10 +61,17 @@ npm install synaptic --save
 ```

 #####In the browser
-Just include the file synaptic.js from `/dist` directory with a script tag in your HTML:
+
+You can install synaptic with [bower](http://bower.io):
+
+```cmd
+bower install synaptic
+```
+
+Or you can simply use the CDN link, kindly provided by [CDNjs](https://cdnjs.com/)

 ```html
-<script src="synaptic.js"></script>
+<script src="https://cdnjs.cloudflare.com/ajax/libs/synaptic/1.0.8/synaptic.js"></script>
 ```

 ###Usage
@@ -68,15 +87,6 @@ var Neuron = synaptic.Neuron,

 Now you can start to create networks, train them, or use built-in networks from the [Architect](http://github.com/cazala/synaptic#architect).

-###Gulp Tasks
-
- **gulp**: runs all the tests and builds the minified and unminified bundles into `/dist`.
- **gulp build**: builds the bundle: `/dist/synaptic.js`.
- **gulp min**: builds the minified bundle: `/dist/synaptic.min.js`.
- **gulp debug**: builds the bundle `/dist/synaptic.js` with sourcemaps.
- **gulp dev**: same as `gulp debug`, but watches the source files and rebuilds when any change is detected.
- **gulp test**: runs all the tests.
-
 ###Examples

 #####Perceptron
@@ -186,6 +196,6 @@ Multilayer LSTM network architectures.

 **Synaptic** is an Open Source project that started in Buenos Aires, Argentina. Anybody in the world is welcome to contribute to the development of the project.

-If you want to contribute feel free to send PR's, just make sure to run the default **gulp** task before submiting it. This way you'll run all the test specs and build the web distribution files.
+If you want to contribute feel free to send PR's, just make sure to run **npm run test** and **npm run build** before submiting it. This way you'll run all the test specs and build the web distribution files.

 <3
@@ -0,0 +1,33 @@
+{
+  "name": "synaptic",
+  "version": "1.0.8",
+  "homepage": "https://github.com/cazala/synaptic",
+  "authors": [
+    "Juan Cazala <juancazala@gmail.com>"
+  ],
+  "description": "architecture-free neural network library for node.js and the browser",
+  "main": "./dist/synaptic.min.js",
+  "moduleType": [
+    "amd",
+    "globals",
+    "node"
+  ],
+  "keywords": [
+    "neural",
+    "network",
+    "deep",
+    "learning",
+    "machine",
+    "learning",
+    "lstm",
+    "perceptron"
+  ],
+  "license": "MIT",
+  "ignore": [
+    "**/.*",
+    "node_modules",
+    "bower_components",
+    "test",
+    "tests"
+  ]
+}
@@ -1,61 +0,0 @@
-'use strict';
-
-var license = '/*\n\nThe MIT License (MIT)\n\nCopyright (c) 2014 Juan Cazala - juancazala.com\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the "Software"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE\n\n\n\n********************************************************************************************\n                                         SYNAPTIC\n********************************************************************************************\n\nSynaptic is a javascript neural network library for node.js and the browser, its generalized\nalgorithm is architecture-free, so you can build and train basically any type of first order\nor even second order neural network architectures.\n\nhttp://en.wikipedia.org/wiki/Recurrent_neural_network#Second_Order_Recurrent_Neural_Network\n\nThe library includes a few built-in architectures like multilayer perceptrons, multilayer\nlong-short term memory networks (LSTM) or liquid state machines, and a trainer capable of\ntraining any given network, and includes built-in training tasks/tests like solving an XOR,\npassing a Distracted Sequence Recall test or an Embeded Reber Grammar test.\n\nThe algorithm implemented by this library has been taken from Derek D. Monner\'s paper:\n\n\nA generalized LSTM-like training algorithm for second-order recurrent neural networks\nhttp://www.overcomplete.net/papers/nn2012.pdf\n\nThere are references to the equations in that paper commented through the source code.\n\n\n********************************************************************************************/\n'
-var globals = 'var Neuron = synaptic.Neuron, Layer = synaptic.Layer, Network = synaptic.Network, Trainer = synaptic.Trainer, Architect = synaptic.Architect;';
-
-// import
-var gulp = require('gulp');
-var browserify = require('browserify');
-var uglify = require('gulp-uglify');
-var mocha = require('gulp-mocha');
-var prepend = require('gulp-insert').prepend;
-var append = require('gulp-insert').append;
-var source = require('vinyl-source-stream');
-var buffer = require('vinyl-buffer');
-
-// default task: runs all the tests, and builds all the files into dist (minified and unminifed)
-gulp.task('default', ['test', 'build', 'min']);
-
-// build source into /dist for the web
-gulp.task('build', function () {
-  return browserify({ entries: ['./src/synaptic.js'] })
-    .bundle()
-    .pipe(source('synaptic.js'))
-    .pipe(buffer())
-    .pipe(append(globals))
-    .pipe(gulp.dest('./dist'));
-});
-
-// build source into /dist for web (minified)
-gulp.task('min', function () {
-  return browserify({ entries: ['./src/synaptic.js'] })
-    .bundle()
-    .pipe(source('synaptic.min.js'))
-    .pipe(buffer())
-    .pipe(uglify())
-    .pipe(prepend(license))
-    .pipe(append(globals))
-    .pipe(gulp.dest('./dist'));
-});
-
-// build source into /dist with sourcemaps for debugging
-gulp.task('debug', function () {
-  return browserify({ entries: ['./src/synaptic.js'], debug: true })
-    .bundle()
-    .pipe(source('synaptic.js'))
-    .pipe(buffer())
-    .pipe(prepend(license))
-    .pipe(append(globals))
-    .pipe(gulp.dest('./dist'));
-});
-
-// run all the tests with mocha
-gulp.task('test', function () {
-    return gulp.src('test/synaptic.js', {read: false})
-        .pipe(mocha());
-});
-
-// watch for changed and re-build (debug)
-gulp.task('dev', function () {
-   gulp.watch('./src/*.js', ['debug']);
-});
@@ -0,0 +1,25 @@
+// Karma configuration
+
+module.exports = function(config) {
+  config.set({
+    basePath: '',
+    frameworks: ['mocha'],
+    files: [
+      'dist/synaptic.js',
+      'test/[^_]*.js'
+    ],
+    exclude: [
+    ],
+    preprocessors: {
+      'test/*.js': ['webpack'],
+    },
+    reporters: ['progress'],
+    port: 9876,
+    colors: true,
+    logLevel: config.LOG_INFO,
+    autoWatch: true,
+    singleRun: false,
+    concurrency: Infinity,
+    browserNoActivityTimeout: 60000,
+  })
+}
@@ -1,22 +1,36 @@
 {
  "name": "synaptic",
-  "version": "0.1.7",
-  "description": "Architecture-free neural network library",
+  "version": "1.0.9",
+  "description": "architecture-free neural network library",
  "main": "./src/synaptic",
  "scripts": {
-    "test": "mocha test"
+    "test": "npm run test:src",
+    "test:src": "mocha test --require src/synaptic.js ./test",
+    "test:dist": "npm run build && npm run test:mocha:dist && npm run test:karma:browsers",
+    "test:mocha:src": "mocha test --require src/synaptic.js ./test",
+    "test:mocha:dist": "mocha test --require dist/synaptic.js ./test",
+    "test:karma:browsers": "karma start --single-run --browsers Chrome,Firefox,SafariPrivate",
+    "test:karma:phantomjs": "karma start --single-run --browsers PhantomJS",
+    "test:travis": "npm run test:mocha:src && npm run build && npm run test:mocha:dist",
+    "build": "webpack --config webpack.config.js"
  },
+  "prepush": [
+    "test",
+    "build"
+  ],
  "devDependencies": {
-    "browserify": "^10.1.3",
-    "gulp": "^3.8.11",
-    "gulp-insert": "^0.4.0",
-    "gulp-mocha": "^2.0.1",
-    "gulp-sourcemaps": "^1.5.2",
-    "gulp-uglify": "^1.2.0",
-    "gulp-util": "^3.0.4",
-    "vinyl-buffer": "^1.0.0",
-    "vinyl-source-stream": "^1.1.0",
-    "mocha": "^2.2.4"
+    "chai": "^3.5.0",
+    "chai-stats": "^0.3.0",
+    "karma": "^1.1.2",
+    "karma-chrome-launcher": "^1.0.1",
+    "karma-firefox-launcher": "^1.0.0",
+    "karma-mocha": "^1.1.1",
+    "karma-phantomjs-launcher": "^1.0.1",
+    "karma-safari-launcher": "^1.0.0",
+    "karma-webpack": "^1.7.0",
+    "mocha": "^2.2.4",
+    "pre-push": "^0.1.1",
+    "webpack": "^1.13.1"
  },
  "repository": {
    "type": "git",
@@ -38,5 +52,7 @@
    "url": "https://github.com/cazala/synaptic/issues"
  },
  "homepage": "http://synaptic.juancazala.com",
-  "engines" : { "node" : "^0.10.0" }
+  "engines": {
+    "node": ">=4"
+  }
 }
@@ -0,0 +1,21 @@
+// update license year and version
+var fs = require('fs')
+module.exports = function() {
+  var year = (new Date).getFullYear()
+  var version = require('./package.json').version
+  // LICENSE
+  var license = fs.readFileSync('LICENSE', 'utf-8')
+  .replace(/\(c\) ([0-9]+)/, `(c) ${year}`)
+  .replace(/SYNAPTIC \(v(.*)\)/, `SYNAPTIC (v${version})`)
+  fs.writeFileSync('LICENSE', license)
+  // bower.json
+  var bower = fs.readFileSync('bower.json', 'utf-8')
+  .replace(/\"version\": \"(.*)\",/, `"version": "${version}",`)
+  fs.writeFileSync('bower.json', bower)
+  // README.md
+  var readme = fs.readFileSync('README.md', 'utf-8')
+  .replace(/ajax\/libs\/synaptic\/(.*)\/synaptic.js/, `ajax/libs/synaptic/${version}/synaptic.js`)
+  fs.writeFileSync('README.md', readme)
+  // return license for dist banner
+  return license
+}
@@ -1,13 +1,13 @@
 // import
-var Layer = require('./layer'),
-    Network = require('./network'),
-    Trainer = require('./trainer');
+var Layer   = require('./layer')
+,   Network = require('./network')
+,   Trainer = require('./trainer')

 /*******************************************************************************************
                                        ARCHITECT
 *******************************************************************************************/

-// Colection of useful built-in architectures
+// Collection of useful built-in architectures
 var Architect = {

  // Multilayer Perceptron
@@ -15,7 +15,7 @@ var Architect = {

    var args = Array.prototype.slice.call(arguments); // convert arguments to Array
    if (args.length < 3)
-      throw "Error: not enough layers (minimum 3) !!";
+      throw new Error("not enough layers (minimum 3) !!");

    var inputs = args.shift(); // first argument
    var outputs = args.pop(); // last argument
@@ -28,7 +28,7 @@ var Architect = {
    var previous = input;

    // generate hidden layers
-    for (level in layers) {
+    for (var level in layers) {
      var size = layers[level];
      var layer = new Layer(size);
      hidden.push(layer);
@@ -53,28 +53,28 @@ var Architect = {

    var args = Array.prototype.slice.call(arguments); // convert arguments to array
    if (args.length < 3)
-      throw "Error: not enough layers (minimum 3) !!";
+      throw new Error("not enough layers (minimum 3) !!");

    var last = args.pop();
    var option = {
      peepholes: Layer.connectionType.ALL_TO_ALL,
-      hiddentohidden: false,
-      outtohidden: false,
-      outtogates: false,
-      intoout: true,
+      hiddenToHidden: false,
+      outputToHidden: false,
+      outputToGates: false,
+      inputToOutput: true,
    };
    if (typeof last != 'number') {
      var outputs = args.pop();
      if (last.hasOwnProperty('peepholes'))
        option.peepholes = last.peepholes;
-      if (last.hasOwnProperty('hiddentohidden'))
-        option.hiddentohidden = last.hiddentohidden;
-      if (last.hasOwnProperty('outtohidden'))
-        option.outtohidden = last.outtohidden;
-      if (last.hasOwnProperty('outtogates'))
-        option.outtogates = last.outtogates;
-      if (last.hasOwnProperty('intoout'))
-        option.intoout = last.intoout;
+      if (last.hasOwnProperty('hiddenToHidden'))
+        option.hiddenToHidden = last.hiddenToHidden;
+      if (last.hasOwnProperty('outputToHidden'))
+        option.outputToHidden = last.outputToHidden;
+      if (last.hasOwnProperty('outputToGates'))
+        option.outputToGates = last.outputToGates;
+      if (last.hasOwnProperty('inputToOutput'))
+        option.inputToOutput = last.inputToOutput;
    } else
      var outputs = last;

@@ -129,20 +129,20 @@ var Architect = {
      var self = memoryCell.project(memoryCell);

      // hidden to hidden recurrent connection
-      if (option.hiddentohidden)
+      if (option.hiddenToHidden)
        memoryCell.project(memoryCell, Layer.connectionType.ALL_TO_ELSE);

      // out to hidden recurrent connection
-      if (option.outtohidden)
+      if (option.outputToHidden)
        outputLayer.project(memoryCell);

      // out to gates recurrent connection
-      if (option.outtogates) {
+      if (option.outputToGates) {
        outputLayer.project(inputGate);
        outputLayer.project(outputGate);
        outputLayer.project(forgetGate);
      }
-      
+
      // peepholes
      memoryCell.project(inputGate, option.peepholes);
      memoryCell.project(forgetGate, option.peepholes);
@@ -159,7 +159,7 @@ var Architect = {
    }

    // input to output direct connection
-    if (option.intoout)
+    if (option.inputToOutput)
      inputLayer.project(outputLayer);

    // set the layers of the neural network
@@ -217,8 +217,8 @@ var Architect = {
    this.trainer = new Trainer(this);
  },

-  Hopfield: function Hopfield(size)
-  {
+  Hopfield: function Hopfield(size) {
+
    var inputLayer = new Layer(size);
    var outputLayer = new Layer(size);

@@ -248,7 +248,7 @@ var Architect = {
        error: .00005,
        rate: 1
      });
-    }
+    };

    proto.feed = proto.feed || function(pattern)
    {
@@ -270,5 +270,4 @@ for (var architecture in Architect) {
 }

 // export
-if (module) module.exports = Architect; 
-
+if (module) module.exports = Architect;
@@ -1,5 +1,9 @@
+// export
+if (module) module.exports = Layer;
+
 // import
-var Neuron = require('./neuron');
+var Neuron  = require('./neuron')
+,   Network = require('./network')

 /*******************************************************************************************
                                            LAYER
@@ -9,7 +13,7 @@ function Layer(size, label) {
  this.size = size | 0;
  this.list = [];
  this.label = label || null;
-  this.connectedto = [];
+  this.connectedTo = [];

  while (size--) {
    var neuron = new Neuron();
@@ -26,7 +30,7 @@ Layer.prototype = {

    if (typeof input != 'undefined') {
      if (input.length != this.size)
-        throw "INPUT size and LAYER size must be the same to activate!";
+        throw new Error("INPUT size and LAYER size must be the same to activate!");

      for (var id in this.list) {
        var neuron = this.list[id];
@@ -48,7 +52,7 @@ Layer.prototype = {

    if (typeof target != 'undefined') {
      if (target.length != this.size)
-        throw "TARGET size and LAYER size must be the same to propagate!";
+        throw new Error("TARGET size and LAYER size must be the same to propagate!");

      for (var id = this.list.length - 1; id >= 0; id--) {
        var neuron = this.list[id];
@@ -65,14 +69,14 @@ Layer.prototype = {
  // projects a connection from this layer to another one
  project: function(layer, type, weights) {

-    if (layer instanceof require('./network'))
+    if (layer instanceof Network)
      layer = layer.layers.input;

    if (layer instanceof Layer) {
      if (!this.connected(layer))
        return new Layer.connection(this, layer, type, weights);
    } else
-      throw "Invalid argument, you can only project connections to LAYERS and NETWORKS!";
+      throw new Error("Invalid argument, you can only project connections to LAYERS and NETWORKS!");


  },
@@ -82,7 +86,7 @@ Layer.prototype = {

    if (type == Layer.gateType.INPUT) {
      if (connection.to.size != this.size)
-        throw "GATER layer and CONNECTION.TO layer must be the same size in order to gate!";
+        throw new Error("GATER layer and CONNECTION.TO layer must be the same size in order to gate!");

      for (var id in connection.to.list) {
        var neuron = connection.to.list[id];
@@ -95,7 +99,7 @@ Layer.prototype = {
      }
    } else if (type == Layer.gateType.OUTPUT) {
      if (connection.from.size != this.size)
-        throw "GATER layer and CONNECTION.FROM layer must be the same size in order to gate!";
+        throw new Error("GATER layer and CONNECTION.FROM layer must be the same size in order to gate!");

      for (var id in connection.from.list) {
        var neuron = connection.from.list[id];
@@ -108,7 +112,7 @@ Layer.prototype = {
      }
    } else if (type == Layer.gateType.ONE_TO_ONE) {
      if (connection.size != this.size)
-        throw "The number of GATER UNITS must be the same as the number of CONNECTIONS to gate!";
+        throw new Error("The number of GATER UNITS must be the same as the number of CONNECTIONS to gate!");

      for (var id in connection.list) {
        var gater = this.list[id];
@@ -248,8 +252,8 @@ Layer.connection = function LayerConnection(fromLayer, toLayer, type, weights) {
      this.size = this.list.push(connection);
    }
  }
-  
-  fromLayer.connectedto.push(this);
+
+  fromLayer.connectedTo.push(this);
 }

 // types of connections
@@ -270,7 +274,3 @@ Layer.gateType.ONE_TO_ONE = "ONE TO ONE";
    return connections++;
  }
 })();
-
-// export
-if (module) module.exports = Layer;
-
@@ -1,6 +1,10 @@
+// export
+if (module) module.exports = Network;
+
 // import
-var Neuron = require('./neuron'),
-    Layer = require('./layer');
+var Neuron  = require('./neuron')
+,   Layer   = require('./layer')
+,   Trainer = require('./trainer')

 /*******************************************************************************************
                                         NETWORK
@@ -27,8 +31,8 @@ Network.prototype = {
      for (var layer in this.layers.hidden)
        this.layers.hidden[layer].activate();
      return this.layers.output.activate();
-    } 
-    else 
+    }
+    else
    {
      if (this.optimized == null)
        this.optimize();
@@ -48,8 +52,8 @@ Network.prototype = {
      reverse.reverse();
      for (var layer in reverse)
        reverse[layer].propagate(rate);
-    } 
-    else 
+    }
+    else
    {
      if (this.optimized == null)
        this.optimize();
@@ -69,7 +73,7 @@ Network.prototype = {
    if (unit instanceof Layer)
      return this.layers.output.project(unit, type, weights);

-    throw "Invalid argument, you can only project connections to LAYERS and NETWORKS!";
+    throw new Error("Invalid argument, you can only project connections to LAYERS and NETWORKS!");
  },

  // let this network gate a connection
@@ -142,7 +146,6 @@ Network.prototype = {
      hardcode += "F[" + optimized.variables[i].id + "] = " + (optimized.variables[
        i].value || 0) + "; ";
    hardcode += "var activate = function(input){\n";
-    hardcode += "influences = [];";
    for (var i in optimized.inputs)
      hardcode += "F[" + optimized.inputs[i] + "] = input[" + i + "]; ";
    for (var currentLayer in optimized.activation_sentences) {
@@ -356,21 +359,6 @@ Network.prototype = {
      neurons.push(copy);
    }

-    if (!ignoreTraces)
-      for (var i in neurons) {
-        var copy = neurons[i];
-
-        for (var input in neuron.trace.elegibility)
-          copy.trace.elegibility[input] = neuron.trace.elegibility[input];
-
-        for (var gated in neuron.trace.extended) {
-          copy.trace.extended[gated] = {};
-          for (var input in neuron.trace.extended[gated])
-            copy.trace.extended[ids[gated]][input] = neuron.trace.extended[
-              gated][input];
-        }
-      }
-
    // get connections
    for (var i in list) {
      var neuron = list[i].neuron;
@@ -391,8 +379,7 @@ Network.prototype = {
          from: ids[neuron.ID],
          to: ids[neuron.ID],
          weight: neuron.selfconnection.weight,
-          gater: neuron.selfconnection.gater ? ids[neuron.selfconnection.gater
-            .ID] : null,
+          gater: neuron.selfconnection.gater ? ids[neuron.selfconnection.gater.ID] : null,
        });
    }

@@ -401,50 +388,48 @@ Network.prototype = {
      connections: connections
    }
  },
-  
+
  // export the topology into dot language which can be visualized as graphs using dot
  /* example: ... console.log(net.toDotLang());
              $ node example.js > example.dot
              $ dot example.dot -Tpng > out.png
  */
-  toDot: function(edgeconnection) {
-    if (! typeof edgeconnection)
-      edgeconnection = false;
+  toDot: function(edgeConnection) {
+    if (! typeof edgeConnection)
+      edgeConnection = false;
    var code = "digraph nn {\n    rankdir = BT\n";
    var layers = [this.layers.input].concat(this.layers.hidden, this.layers.output);
    for (var layer in layers) {
-      for (var to in layers[layer].connectedto) { // projections
-        var connection = layers[layer].connectedto[to];
-        var layerto = connection.to;
+      for (var to in layers[layer].connectedTo) { // projections
+        var connection = layers[layer].connectedTo[to];
+        var layerTo = connection.to;
        var size = connection.size;
        var layerID = layers.indexOf(layers[layer]);
-        var layertoID = layers.indexOf(layerto);
+        var layerToID = layers.indexOf(layerTo);
        /* http://stackoverflow.com/questions/26845540/connect-edges-with-graph-dot
         * DOT does not support edge-to-edge connections
         * This workaround produces somewhat weird graphs ...
        */
-        if ( edgeconnection) {
+        if ( edgeConnection) {
          if (connection.gatedfrom.length) {
-            var fakeNode = "fake" + layerID + "_" + layertoID;
+            var fakeNode = "fake" + layerID + "_" + layerToID;
            code += "    " + fakeNode +
              " [label = \"\", shape = point, width = 0.01, height = 0.01]\n";
            code += "    " + layerID + " -> " + fakeNode + " [label = " + size + ", arrowhead = none]\n";
-            code += "    " + fakeNode + " -> " + layertoID + "\n";
+            code += "    " + fakeNode + " -> " + layerToID + "\n";
          } else
-            code += "    " + layerID + " -> " + layertoID + " [label = " + size + "]\n";
+            code += "    " + layerID + " -> " + layerToID + " [label = " + size + "]\n";
          for (var from in connection.gatedfrom) { // gatings
            var layerfrom = connection.gatedfrom[from].layer;
-            var type = connection.gatedfrom[from].type;
            var layerfromID = layers.indexOf(layerfrom);
            code += "    " + layerfromID + " -> " + fakeNode + " [color = blue]\n";
          }
        } else {
-          code += "    " + layerID + " -> " + layertoID + " [label = " + size + "]\n";
+          code += "    " + layerID + " -> " + layerToID + " [label = " + size + "]\n";
          for (var from in connection.gatedfrom) { // gatings
            var layerfrom = connection.gatedfrom[from].layer;
-            var type = connection.gatedfrom[from].type;
            var layerfromID = layers.indexOf(layerfrom);
-            code += "    " + layerfromID + " -> " + layertoID + " [color = blue]\n";
+            code += "    " + layerfromID + " -> " + layerToID + " [color = blue]\n";
          }
        }
      }
@@ -473,7 +458,7 @@ Network.prototype = {
    // build network activation
    for (var neuron in data.activate) { // shouldn't this be layer?
      for (var sentence in data.activate[neuron])
-        activation += data.activate[neuron][sentence] + "\n";
+        activation += data.activate[neuron][sentence].join('') + "\n";
    }

    // build outputs
@@ -506,44 +491,110 @@ Network.prototype = {
    return new Function(hardcode)();
  },

-  worker: function() {
+
+  // Return a HTML5 WebWorker specialized on training the network stored in `memory`.
+  // Train based on the given dataSet and options.
+  // The worker returns the updated `memory` when done.
+  worker: function(memory, set, options) {
+
+    // Copy the options and set defaults (options might be different for each worker)
+    var workerOptions = {};
+    if(options) workerOptions = options;
+    workerOptions.rate = options.rate || .2;
+    workerOptions.iterations = options.iterations || 100000;
+    workerOptions.error = options.error || .005;
+    workerOptions.cost = options.cost || null;
+    workerOptions.crossValidate = options.crossValidate || null;
+
+    // Cost function might be different for each worker
+    costFunction = "var cost = " + (options && options.cost || this.cost || Trainer.cost.MSE) + ";\n";
+    var workerFunction = Network.getWorkerSharedFunctions();
+    workerFunction = workerFunction.replace(/var cost = options && options\.cost \|\| this\.cost \|\| Trainer\.cost\.MSE;/g, costFunction);
+
+    // Set what we do when training is finished
+    workerFunction = workerFunction.replace('return results;',
+                      'postMessage({action: "done", message: results, memoryBuffer: F}, [F.buffer]);');
+
+    // Replace log with postmessage
+    workerFunction = workerFunction.replace("console.log('iterations', iterations, 'error', error, 'rate', currentRate)",
+              "postMessage({action: 'log', message: {\n" +
+                  "iterations: iterations,\n" +
+                  "error: error,\n" +
+                  "rate: currentRate\n" +
+                "}\n" +
+              "})");
+
+    // Replace schedule with postmessage
+    workerFunction = workerFunction.replace("abort = this.schedule.do({ error: error, iterations: iterations, rate: currentRate })",
+              "postMessage({action: 'schedule', message: {\n" +
+                  "iterations: iterations,\n" +
+                  "error: error,\n" +
+                  "rate: currentRate\n" +
+                "}\n" +
+              "})");
+
    if (!this.optimized)
      this.optimize();

-    var hardcode = "var inputs = " + this.optimized.data.inputs.length +
-      ";\n";
-    hardcode += "var outputs = " + this.optimized.data.outputs.length +
-      ";\n";
-    hardcode += "var F = null;\n";
-    hardcode += "var activate = " + this.optimized.activate.toString() +
-      ";\n";
-    hardcode += "var propagate = " + this.optimized.propagate.toString() +
-      ";\n";
-    hardcode += "onmessage = function(e){\n";
-    hardcode += "F = e.data.memoryBuffer;\n";
-    hardcode += "if (e.data.action == 'activate'){\n";
-    hardcode += "if (e.data.input.length == inputs){\n";
+    var hardcode = "var inputs = " + this.optimized.data.inputs.length + ";\n";
+    hardcode += "var outputs = " + this.optimized.data.outputs.length + ";\n";
+    hardcode += "var F =  new Float64Array([" + this.optimized.memory.toString() + "]);\n";
+    hardcode += "var activate = " + this.optimized.activate.toString() + ";\n";
+    hardcode += "var propagate = " + this.optimized.propagate.toString() + ";\n";
    hardcode +=
-      "postMessage( { action: 'activate', output: activate(e.data.input), memoryBuffer: F }, [F.buffer]);\n";
-    hardcode += "}\n}\nelse if (e.data.action == 'propagate'){\n";
-    hardcode += "propagate(e.data.rate, e.data.target);\n";
-    hardcode +=
-      "postMessage({ action: 'propagate', memoryBuffer: F }, [F.buffer]);\n";
-    hardcode += "}\n}\n";
+        "onmessage = function(e) {\n" +
+          "if (e.data.action == 'startTraining') {\n" +
+            "train(" + JSON.stringify(set) + "," + JSON.stringify(workerOptions) + ");\n" +
+          "}\n" +
+        "}";

-    var blob = new Blob([hardcode]);
+    var workerSourceCode = workerFunction + '\n' + hardcode;
+    var blob = new Blob([workerSourceCode]);
    var blobURL = window.URL.createObjectURL(blob);

    return new Worker(blobURL);
  },

  // returns a copy of the network
-  clone: function(ignoreTraces) {
-    return Network.fromJSON(this.toJSON(ignoreTraces));
+  clone: function() {
+    return Network.fromJSON(this.toJSON());
  }
-}
+};

-// rebuild a network that has been stored in a json using the method toJson()
+/**
+ * Creates a static String to store the source code of the functions
+ *  that are identical for all the workers (train, _trainSet, test)
+ *
+ * @return {String} Source code that can train a network inside a worker.
+ * @static
+ */
+Network.getWorkerSharedFunctions = function() {
+  // If we already computed the source code for the shared functions
+  if(typeof Network._SHARED_WORKER_FUNCTIONS !== 'undefined')
+    return Network._SHARED_WORKER_FUNCTIONS;
+
+  // Otherwise compute and return the source code
+  // We compute them by simply copying the source code of the train, _trainSet and test functions
+  //  using the .toString() method
+
+  // Load and name the train function
+  var train_f = Trainer.prototype.train.toString();
+  train_f = train_f.replace('function (set', 'function train(set') + '\n';
+
+  // Load and name the _trainSet function
+  var _trainSet_f = Trainer.prototype._trainSet.toString().replace(/this.network./g, '');
+  _trainSet_f = _trainSet_f.replace('function (set', 'function _trainSet(set') + '\n';
+  _trainSet_f = _trainSet_f.replace('this.crossValidate', 'crossValidate');
+  _trainSet_f = _trainSet_f.replace('crossValidate = true', 'crossValidate = { }');
+
+  // Load and name the test function
+  var test_f = Trainer.prototype.test.toString().replace(/this.network./g, '');
+  test_f = test_f.replace('function (set', 'function test(set') + '\n';
+
+  return Network._SHARED_WORKER_FUNCTIONS = train_f + _trainSet_f + test_f;
+};
+
+// rebuild a network that has been stored in a json using the method toJSON()
 Network.fromJSON = function(json) {

  var neurons = [];
@@ -552,20 +603,19 @@ Network.fromJSON = function(json) {
    input: new Layer(),
    hidden: [],
    output: new Layer()
-  }
+  };

  for (var i in json.neurons) {
    var config = json.neurons[i];

    var neuron = new Neuron();
-    neuron.trace.elegibility = config.trace.elegibility;
-    neuron.trace.extended = config.trace.extended;
+    neuron.trace.elegibility = {};
+    neuron.trace.extended = {};
    neuron.state = config.state;
    neuron.old = config.old;
    neuron.activation = config.activation;
    neuron.bias = config.bias;
-    neuron.squash = config.squash in Neuron.squash ? Neuron.squash[config.squash] :
-      Neuron.squash.LOGISTIC;
+    neuron.squash = config.squash in Neuron.squash ? Neuron.squash[config.squash] : Neuron.squash.LOGISTIC;
    neurons.push(neuron);

    if (config.layer == 'input')
@@ -583,7 +633,7 @@ Network.fromJSON = function(json) {
    var config = json.connections[i];
    var from = neurons[config.from];
    var to = neurons[config.to];
-    var weight = config.weight
+    var weight = config.weight;
    var gater = neurons[config.gater];

    var connection = from.project(to, weight);
@@ -592,8 +642,4 @@ Network.fromJSON = function(json) {
  }

  return new Network(layers);
-}
-
-// export
-if (module) module.exports = Network;
-
+};
@@ -1,3 +1,6 @@
+// export
+if (module) module.exports = Neuron;
+
 /******************************************************************************************
                                         NEURON
 *******************************************************************************************/
@@ -63,7 +66,6 @@ Neuron.prototype = {
    var influences = [];
    for (var id in this.trace.extended) {
      // extended elegibility trace
-      var xtrace = this.trace.extended[id];
      var neuron = this.neighboors[id];

      // if gated neuron's selfconnection is gated by this unit, the influence keeps track of the neuron's old state
@@ -76,7 +78,7 @@ Neuron.prototype = {
      }
      influences[neuron.ID] = influence;
    }
-    
+
    for (var i in this.connections.inputs) {
      var input = this.connections.inputs[i];

@@ -117,7 +119,7 @@ Neuron.prototype = {
    // output neurons get their error from the enviroment
    if (isOutput)
      this.error.responsibility = this.error.projected = target - this.activation; // Eq. 10
-    
+
    else // the rest of the neuron compute their error responsibilities by backpropagation
    {
      // error responsibilities from all the connections projected from this neuron
@@ -299,9 +301,8 @@ Neuron.prototype = {

  // hardcodes the behaviour of the neuron into an optimized function
  optimize: function(optimized, layer) {
-    
+
    optimized = optimized || {};
-    var that = this;
    var store_activation = [];
    var store_trace = [];
    var store_propagation = [];
@@ -324,7 +325,7 @@ Neuron.prototype = {
        layers.__count = store.push([]) - 1;
        layers[layer] = layers.__count;
      }
-    }
+    };
    allocate(activation_sentences);
    allocate(trace_sentences);
    allocate(propagation_sentences);
@@ -383,7 +384,7 @@ Neuron.prototype = {
          sentence += 'F[' + args[i].id + ']';

      store.push(sentence + ';');
-    }
+    };

    // helper to check if an object is empty
    var isEmpty = function(obj) {
@@ -460,19 +461,19 @@ Neuron.prototype = {
          buildSentence(derivative, ' = 1', store_activation);
          break;
        case Neuron.squash.HLIM:
-          buildSentence(activation, ' = +(', state, ' > 0)',
-            store_activation);
+          buildSentence(activation, ' = +(', state, ' > 0)', store_activation);
          buildSentence(derivative, ' = 1', store_activation);
+        case Neuron.squash.RELU:
+          buildSentence(activation, ' = ', state, ' > 0 ? ', state, ' : 0', store_activation);
+          buildSentence(derivative, ' = ', state, ' > 0 ? 1 : 0', store_activation);
          break;
      }

-      influences = [];
      for (var id in this.trace.extended) {
        // calculate extended elegibility traces in advance
-        
-        var xtrace = this.trace.extended[id];
+
        var neuron = this.neighboors[id];
-        var influence = getVar('aux');
+        var influence = getVar('influences[' + neuron.ID + ']');
        var neuron_old = getVar(neuron, 'old');
        var initialized = false;
        if (neuron.selfconnection.gater == this)
@@ -487,19 +488,14 @@ Neuron.prototype = {
            [incoming].from, 'activation');

          if (initialized)
-            buildSentence(influence, ' += ', incoming_weight, ' * ',
-              incoming_activation, store_trace);
+            buildSentence(influence, ' += ', incoming_weight, ' * ', incoming_activation, store_trace);
          else {
-            buildSentence(influence, ' = ', incoming_weight, ' * ',
-              incoming_activation, store_trace);
+            buildSentence(influence, ' = ', incoming_weight, ' * ', incoming_activation, store_trace);
            initialized = true;
          }
        }
-        
-        influences.push(neuron.ID);
-        buildSentence("influences[" + (influences.length - 1) + "] = ", influence, store_trace);
      }
-      
+
      for (var i in this.connections.inputs) {
        var input = this.connections.inputs[i];
        if (input.gater)
@@ -533,10 +529,8 @@ Neuron.prototype = {
        }
        for (var id in this.trace.extended) {
          // extended elegibility trace
-          var xtrace = this.trace.extended[id];
          var neuron = this.neighboors[id];
-          var influence = getVar('aux');
-          var neuron_old = getVar(neuron, 'old');
+          var influence = getVar('influences[' + neuron.ID + ']');

          var trace = getVar(this, 'trace', 'elegibility', input.ID, this.trace
            .elegibility[input.ID]);
@@ -550,14 +544,14 @@ Neuron.prototype = {
            if (neuron.selfconnection.gater)
              buildSentence(xtrace, ' = ', neuron_self_gain, ' * ',
                neuron_self_weight, ' * ', xtrace, ' + ', derivative, ' * ',
-                trace, ' * ', "influences[" + influences.indexOf(neuron.ID) + "]", store_trace);
+                trace, ' * ', influence, store_trace);
            else
              buildSentence(xtrace, ' = ', neuron_self_weight, ' * ',
                xtrace, ' + ', derivative, ' * ', trace, ' * ',
-                "influences[" + influences.indexOf(neuron.ID) + "]", store_trace);
+                influence, store_trace);
          else
            buildSentence(xtrace, ' = ', derivative, ' * ', trace, ' * ',
-              "influences[" + influences.indexOf(neuron.ID) + "]", store_trace);
+              influence, store_trace);
        }
      }
      for (var connection in this.connections.gated) {
@@ -742,7 +736,7 @@ Neuron.prototype = {
 Neuron.connection = function Connection(from, to, weight) {

  if (!from || !to)
-    throw "Connection Error: Invalid neurons";
+    throw new Error("Connection Error: Invalid neurons");

  this.ID = Neuron.connection.uid();
  this.from = from;
@@ -775,7 +769,12 @@ Neuron.squash.IDENTITY = function(x, derivate) {
  return derivate ? 1 : x;
 };
 Neuron.squash.HLIM = function(x, derivate) {
-  return derivate ? 1 : +(x > 0);
+  return derivate ? 1 : x > 0 ? 1 : 0;
+};
+Neuron.squash.RELU = function(x, derivate) {
+  if (derivate)
+    return x > 0 ? 1 : 0;
+  return x > 0 ? x : 0;
 };

 // unique ID's
@@ -795,7 +794,3 @@ Neuron.squash.HLIM = function(x, derivate) {
    }
  }
 })();
-
-// export
-if (module) module.exports = Neuron;
-
@@ -1,54 +1,3 @@
-/*
-
-The MIT License (MIT)
-
-Copyright (c) 2014 Juan Cazala - juancazala.com
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in
-all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-THE SOFTWARE
-
-
-
-********************************************************************************************
-                                         SYNAPTIC
-********************************************************************************************
-
-Synaptic is a javascript neural network library for node.js and the browser, its generalized
-algorithm is architecture-free, so you can build and train basically any type of first order
-or even second order neural network architectures.
-
-http://en.wikipedia.org/wiki/Recurrent_neural_network#Second_Order_Recurrent_Neural_Network
-
-The library includes a few built-in architectures like multilayer perceptrons, multilayer
-long-short term memory networks (LSTM) or liquid state machines, and a trainer capable of
-training any given network, and includes built-in training tasks/tests like solving an XOR,
-passing a Distracted Sequence Recall test or an Embeded Reber Grammar test.
-
-The algorithm implemented by this library has been taken from Derek D. Monner's paper:
-
-A generalized LSTM-like training algorithm for second-order recurrent neural networks
-http://www.overcomplete.net/papers/nn2012.pdf
-
-There are references to the equations in that paper commented through the source code.
-
-
-********************************************************************************************/
-
 var Synaptic = {
    Neuron: require('./neuron'),
    Layer: require('./layer'),
@@ -72,12 +21,12 @@ if (typeof module !== 'undefined' && module.exports)
 // Browser
 if (typeof window == 'object')
 {
-  (function(){ 
+  (function(){
    var oldSynaptic = window['synaptic'];
-    Synaptic.ninja = function(){ 
-      window['synaptic'] = oldSynaptic; 
+    Synaptic.ninja = function(){
+      window['synaptic'] = oldSynaptic;
      return Synaptic;
-    };	
+    };
  })();

  window['synaptic'] = Synaptic;
@@ -1,3 +1,6 @@
+// export
+if (module) module.exports = Trainer;
+
 /*******************************************************************************************
                                        TRAINER
 *******************************************************************************************/
@@ -7,8 +10,9 @@ function Trainer(network, options) {
  this.network = network;
  this.rate = options.rate || .2;
  this.iterations = options.iterations || 100000;
-  this.error = options.error || .005
-  this.cost = options.cost || Trainer.cost.CROSS_ENTROPY;
+  this.error = options.error || .005;
+  this.cost = options.cost || null;
+  this.crossValidate = options.crossValidate || null;
 }

 Trainer.prototype = {
@@ -18,8 +22,10 @@ Trainer.prototype = {

    var error = 1;
    var iterations = bucketSize = 0;
-    var abort_training = false;
-    var input, output, target, currentRate;
+    var abort = false;
+    var currentRate;
+    var cost = options && options.cost || this.cost || Trainer.cost.MSE;
+    var crossValidate = false, testSet, trainSet;

    var start = Date.now();

@@ -47,44 +53,63 @@ Trainer.prototype = {
        console.log('Deprecated: use schedule instead of customLog')
        this.schedule = options.customLog;
      }
+      if (this.crossValidate || options.crossValidate) {
+        if(!this.crossValidate) this.crossValidate = {};
+        crossValidate = true;
+        if (options.crossValidate.testSize)
+          this.crossValidate.testSize = options.crossValidate.testSize;
+        if (options.crossValidate.testError)
+          this.crossValidate.testError = options.crossValidate.testError;
+      }
    }

    currentRate = this.rate;
    if(Array.isArray(this.rate)) {
-      bucketSize = Math.floor(this.iterations / this.rate.length);
+      var bucketSize = Math.floor(this.iterations / this.rate.length);
    }

+    if(crossValidate) {
+      var numTrain = Math.ceil((1 - this.crossValidate.testSize) * set.length);
+      trainSet = set.slice(0, numTrain);
+      testSet = set.slice(numTrain);
+    }

-    while (!abort_training && iterations < this.iterations && error > this.error) {
+    var lastError = 0;
+    while ((!abort && iterations < this.iterations && error > this.error)) {
+      if (crossValidate && error <= this.crossValidate.testError) {
+        break;
+      }
+
+      var currentSetSize = set.length;
      error = 0;
+      iterations++;

      if(bucketSize > 0) {
        var currentBucket = Math.floor(iterations / bucketSize);
-        currentRate = this.rate[currentBucket];
+        currentRate = this.rate[currentBucket] || currentRate;
+      }
+      
+      if(typeof this.rate === 'function') {
+        currentRate = this.rate(iterations, lastError);
      }

-      for (var train in set) {
-        input = set[train].input;
-        target = set[train].output;
-
-        output = this.network.activate(input);
-        this.network.propagate(currentRate, target);
-
-        error += this.cost(target, output);
+      if (crossValidate) {
+        this._trainSet(trainSet, currentRate, cost);
+        error += this.test(testSet).error;
+        currentSetSize = 1;
+      } else {
+        error += this._trainSet(set, currentRate, cost);
+        currentSetSize = set.length;
      }

      // check error
-      iterations++;
-      error /= set.length;
+      error /= currentSetSize;
+      lastError = error;

      if (options) {
        if (this.schedule && this.schedule.every && iterations %
          this.schedule.every == 0)
-          abort_training = this.schedule.do({
-            error: error,
-            iterations: iterations,
-            rate: currentRate
-          });
+          abort = this.schedule.do({ error: error, iterations: iterations, rate: currentRate });
        else if (options.log && iterations % options.log == 0) {
          console.log('iterations', iterations, 'error', error, 'rate', currentRate);
        };
@@ -97,152 +122,127 @@ Trainer.prototype = {
      error: error,
      iterations: iterations,
      time: Date.now() - start
-    }
+    };

    return results;
  },

-  // trains any given set to a network using a WebWorker
-  workerTrain: function(set, callback, options) {
+  // trains any given set to a network, using a WebWorker (only for the browser). Returns a Promise of the results.
+  trainAsync: function(set, options) {
+    var train = this.workerTrain.bind(this);
+    return new Promise(function(resolve, reject) {
+      try {
+        train(set, resolve, options, true)
+      } catch(e) {
+        reject(e)
+      }
+    })
+  },

-    var that = this;
-    var error = 1;
-    var iterations = bucketSize = 0;
-    var input, output, target, currentRate;
-    var length = set.length;
-    var abort_training = false;
+  // preforms one training epoch and returns the error (private function used in this.train)
+  _trainSet: function(set, currentRate, costFunction) {
+    var errorSum = 0;
+    for (var train in set) {
+      var input = set[train].input;
+      var target = set[train].output;
+
+      var output = this.network.activate(input);
+      this.network.propagate(currentRate, target);
+
+      errorSum += costFunction(target, output);
+    }
+    return errorSum;
+  },
+
+  // tests a set and returns the error and elapsed time
+  test: function(set, options) {
+
+    var error = 0;
+    var input, output, target;
+    var cost = options && options.cost || this.cost || Trainer.cost.MSE;

    var start = Date.now();

-    if (options) {
-      if (options.shuffle) {
-        //+ Jonas Raoni Soares Silva
-        //@ http://jsfromhell.com/array/shuffle [v1.0]
-        function shuffle(o) { //v1.0
-          for (var j, x, i = o.length; i; j = Math.floor(Math.random() *
-              i), x = o[--i], o[i] = o[j], o[j] = x);
-          return o;
-        };
-      }
-      if (options.iterations)
-        this.iterations = options.iterations;
-      if (options.error)
-        this.error = options.error;
-      if (options.rate)
-        this.rate = options.rate;
-      if (options.cost)
-        this.cost = options.cost;
-      if (options.schedule)
-        this.schedule = options.schedule;
-      if (options.customLog)
-        // for backward compatibility with code that used customLog
-        console.log('Deprecated: use schedule instead of customLog')
-        this.schedule = options.customLog;
+    for (var test in set) {
+      input = set[test].input;
+      target = set[test].output;
+      output = this.network.activate(input);
+      error += cost(target, output);
    }

-    // dynamic learning rate
-    currentRate = this.rate;
-    if(Array.isArray(this.rate)) {
-      bucketSize = Math.floor(this.iterations / this.rate.length);
-    }
+    error /= set.length;

-    // create a worker
-    var worker = this.network.worker();
+    var results = {
+      error: error,
+      time: Date.now() - start
+    };

-    // activate the network
-    function activateWorker(input)
-    {
-        worker.postMessage({ 
-            action: "activate",
-            input: input,
-            memoryBuffer: that.network.optimized.memory
-        }, [that.network.optimized.memory.buffer]);
-    }
+    return results;
+  },

-    // backpropagate the network
-    function propagateWorker(target){
-        if(bucketSize > 0) {
-          var currentBucket = Math.floor(iterations / bucketSize);
-          currentRate = this.rate[currentBucket];
-        }
-        worker.postMessage({ 
-            action: "propagate",
-            target: target,
-            rate: currentRate,
-            memoryBuffer: that.network.optimized.memory
-        }, [that.network.optimized.memory.buffer]);
+  // trains any given set to a network using a WebWorker [deprecated: use trainAsync instead]
+  workerTrain: function(set, callback, options, suppressWarning) {
+
+    if (!suppressWarning) {
+      console.warn('Deprecated: do not use `workerTrain`, use `trainAsync` instead.')
    }
+    var that = this;
+
+    if (!this.network.optimized)
+      this.network.optimize();
+
+    // Create a new worker
+    var worker = this.network.worker(this.network.optimized.memory, set, options);

    // train the worker
-    worker.onmessage = function(e){
-        // give control of the memory back to the network
-        that.network.optimized.ownership(e.data.memoryBuffer);
+    worker.onmessage = function(e) {
+      switch(e.data.action) {
+          case 'done':
+            var iterations = e.data.message.iterations;
+            var error = e.data.message.error;
+            var time = e.data.message.time;

-        if (e.data.action == "propagate")
-        {
-            if (index >= length)
-            {
-                index = 0;
-                iterations++;
-                error /= set.length;
+            that.network.optimized.ownership(e.data.memoryBuffer);

-                // log
-                if (options) {
-                  if (this.schedule && this.schedule.every && iterations % this.schedule.every == 0)
-                    abort_training = this.schedule.do({
-                      error: error,
-                      iterations: iterations
-                    });
-                  else if (options.log && iterations % options.log == 0) {
-                    console.log('iterations', iterations, 'error', error);
-                  };
-                  if (options.shuffle)
-                    shuffle(set);
-                }
+            // Done callback
+            callback({
+              error: error,
+              iterations: iterations,
+              time: time
+            });

-                if (!abort_training && iterations < that.iterations && error > that.error)
-                {
-                    activateWorker(set[index].input);
-                } else {
-                    // callback
-                    callback({
-                      error: error,
-                      iterations: iterations,
-                      time: Date.now() - start
-                    })
-                }
-                error = 0;
-            } else {
-                activateWorker(set[index].input);
+            // Delete the worker and all its associated memory
+            worker.terminate();
+          break;
+
+          case 'log':
+            console.log(e.data.message);
+
+          case 'schedule':
+            if (options && options.schedule && typeof options.schedule.do === 'function') {
+              var scheduled = options.schedule.do
+              scheduled(e.data.message)
            }
-        }
+          break;
+      }
+    };

-        if (e.data.action == "activate")
-        {
-            error += that.cost(set[index].output, e.data.output);
-            propagateWorker(set[index].output); 
-            index++;
-        }
-    }
-
-    // kick it
-    var index = 0;
-    var iterations = 0;
-    activateWorker(set[index].input);
+    // Start the worker
+    worker.postMessage({action: 'startTraining'});
  },

  // trains an XOR to the network
  XOR: function(options) {

    if (this.network.inputs() != 2 || this.network.outputs() != 1)
-      throw "Error: Incompatible network (2 inputs, 1 output)";
+      throw new Error("Incompatible network (2 inputs, 1 output)");

    var defaults = {
      iterations: 100000,
      log: false,
      shuffle: true,
      cost: Trainer.cost.MSE
-    }
+    };

    if (options)
      for (var i in options)
@@ -276,9 +276,11 @@ Trainer.prototype = {
    var rate = options.rate || .1;
    var log = options.log || 0;
    var schedule = options.schedule || {};
+    var cost = options.cost || this.cost || Trainer.cost.CROSS_ENTROPY;

-    var trial = correct = i = j = success = 0,
-      error = 1,
+    var trial, correct, i, j, success;
+    trial = correct = i = j = success = 0;
+    var error = 1,
      symbols = targets.length + distractors.length + prompts.length;

    var noRepeat = function(range, avoid) {
@@ -288,14 +290,14 @@ Trainer.prototype = {
        if (number == avoid[i])
          used = true;
      return used ? noRepeat(range, avoid) : number;
-    }
+    };

    var equal = function(prediction, output) {
      for (var i in prediction)
        if (Math.round(prediction[i]) != output[i])
          return false;
      return true;
-    }
+    };

    var start = Date.now();

@@ -320,6 +322,7 @@ Trainer.prototype = {
      }

      //train sequence
+      var distractorsCorrect;
      var targetsCorrect = distractorsCorrect = 0;
      error = 0;
      for (i = 0; i < length; i++) {
@@ -351,10 +354,7 @@ Trainer.prototype = {
          this.network.propagate(rate, output);
        }

-        var delta = 0;
-        for (var j in prediction)
-          delta += Math.pow(output[j] - prediction[j], 2);
-        error += delta / this.network.outputs();
+        error += cost(output, prediction);

        if (distractorsCorrect + targetsCorrect == length)
          correct++;
@@ -399,11 +399,12 @@ Trainer.prototype = {
    var criterion = options.error || .05;
    var rate = options.rate || .1;
    var log = options.log || 500;
+    var cost = options.cost || this.cost || Trainer.cost.CROSS_ENTROPY;

    // gramar node
    var Node = function() {
      this.paths = [];
-    }
+    };
    Node.prototype = {
      connect: function(node, value) {
        this.paths.push({
@@ -424,7 +425,7 @@ Trainer.prototype = {
            return this.paths[i];
        return false;
      }
-    }
+    };

    var reberGrammar = function() {

@@ -433,19 +434,19 @@ Trainer.prototype = {
      var n1 = (new Node()).connect(output, "E");
      var n2 = (new Node()).connect(n1, "S");
      var n3 = (new Node()).connect(n1, "V").connect(n2, "P");
-      var n4 = (new Node()).connect(n2, "X")
+      var n4 = (new Node()).connect(n2, "X");
      n4.connect(n4, "S");
-      var n5 = (new Node()).connect(n3, "V")
+      var n5 = (new Node()).connect(n3, "V");
      n5.connect(n5, "T");
-      n2.connect(n5, "X")
+      n2.connect(n5, "X");
      var n6 = (new Node()).connect(n4, "T").connect(n5, "P");
-      var input = (new Node()).connect(n6, "B")
+      var input = (new Node()).connect(n6, "B");

      return {
        input: input,
        output: output
      }
-    }
+    };

    // build an embeded reber grammar
    var embededReberGrammar = function() {
@@ -465,7 +466,7 @@ Trainer.prototype = {
        output: output
      }

-    }
+    };

    // generate an ERG sequence
    var generate = function() {
@@ -477,7 +478,7 @@ Trainer.prototype = {
        next = next.node.any();
      }
      return str;
-    }
+    };

    // test if a string matches an embeded reber grammar
    var test = function(str) {
@@ -492,7 +493,7 @@ Trainer.prototype = {
        ch = str.charAt(++i);
      }
      return true;
-    }
+    };

    // helper to check if the output and the target vectors match
    var different = function(array1, array2) {
@@ -512,7 +513,7 @@ Trainer.prototype = {
      }

      return i1 != i2;
-    }
+    };

    var iteration = 0;
    var error = 1;
@@ -523,7 +524,7 @@ Trainer.prototype = {
      "X": 3,
      "S": 4,
      "E": 5
-    }
+    };

    var start = Date.now();
    while (iteration < iterations && error > criterion) {
@@ -557,12 +558,7 @@ Trainer.prototype = {
        read = sequence.charAt(++i);
        predict = sequence.charAt(i + 1);

-        var delta = 0;
-        for (var k in output)
-          delta += Math.pow(target[k] - output[k], 2)
-        delta /= output.length;
-
-        error += delta;
+        error += cost(target, output);
      }
      error /= sequence.length;
      iteration++;
@@ -579,6 +575,78 @@ Trainer.prototype = {
      test: test,
      generate: generate
    }
+  },
+
+  timingTask: function(options){
+
+    if (this.network.inputs() != 2 || this.network.outputs() != 1)
+      throw new Error("Invalid Network: must have 2 inputs and one output");
+
+    if (typeof options == 'undefined')
+      options = {};
+
+    // helper
+    function getSamples (trainingSize, testSize){
+
+      // sample size
+      var size = trainingSize + testSize;
+
+      // generate samples
+      var t = 0;
+      var set = [];
+      for (var i = 0; i < size; i++) {
+        set.push({ input: [0,0], output: [0] });
+      }
+      while(t < size - 20) {
+          var n = Math.round(Math.random() * 20);
+          set[t].input[0] = 1;
+          for (var j = t; j <= t + n; j++){
+              set[j].input[1] = n / 20;
+              set[j].output[0] = 0.5;
+          }
+          t += n;
+          n = Math.round(Math.random() * 20);
+          for (var k = t+1; k <= (t + n) &&  k < size; k++)
+              set[k].input[1] = set[t].input[1];
+          t += n;
+      }
+
+      // separate samples between train and test sets
+      var trainingSet = []; var testSet = [];
+      for (var l = 0; l < size; l++)
+          (l < trainingSize ? trainingSet : testSet).push(set[l]);
+
+      // return samples
+      return {
+          train: trainingSet,
+          test: testSet
+      }
+    }
+
+    var iterations = options.iterations || 200;
+    var error = options.error || .005;
+    var rate = options.rate || [.03, .02];
+    var log = options.log === false ? false : options.log || 10;
+    var cost = options.cost || this.cost || Trainer.cost.MSE;
+    var trainingSamples = options.trainSamples || 7000;
+    var testSamples = options.trainSamples || 1000;
+
+    // samples for training and testing
+    var samples = getSamples(trainingSamples, testSamples);
+
+    // train
+    var result = this.train(samples.train, {
+      rate: rate,
+      log: log,
+      iterations: iterations,
+      error: error,
+      cost: cost
+    });
+
+    return {
+      train: result,
+      test: this.test(samples.test)
+    }
  }
 };

@@ -598,9 +666,11 @@ Trainer.cost = {
    for (var i in output)
      mse += Math.pow(target[i] - output[i], 2);
    return mse / output.length;
+  },
+  BINARY: function(target, output){
+    var misses = 0;
+    for (var i in output)
+      misses += Math.round(target[i] * 2) != Math.round(output[i] * 2);
+    return misses;
  }
 }
-
-// export
-if (module) module.exports = Trainer;
-
@@ -1,6 +1,6 @@
-Test using gulp, from root directory:
+Test using mocha, from root directory:

-`gulp test`
+`mocha test`

 To test the web version, start a web server at the root dir of this repo, then use your OS browser. 

@@ -0,0 +1 @@
+global.synaptic = require('../dist/synaptic');
@@ -0,0 +1 @@
+global.synaptic = require('../src/synaptic');
@@ -0,0 +1 @@
+[^_]*.js
@@ -1,48 +1,66 @@
 // import
+var chai = require('chai');
+chai.use(require('chai-stats'));
+var assert = chai.assert;

-var assert = require('assert'),
-  synaptic = require('../src/synaptic');
+var Perceptron = synaptic.Architect.Perceptron;
+var LSTM = synaptic.Architect.LSTM;
+var Layer = synaptic.Layer;
+var Network = synaptic.Network;
+var Trainer = synaptic.Trainer;

-var Perceptron = synaptic.Architect.Perceptron,
-  LSTM = synaptic.Architect.LSTM,
-  Layer = synaptic.Layer,
-  Network = synaptic.Network,
-  Trainer = synaptic.Trainer;
+
+
+var learningRate = .5;


 // utils

-var noRepeat = function(range, avoid) {
+function noRepeat(range, avoid) {
  var number = Math.random() * range | 0;
-  var used = false;
-  for (var i in avoid)
-    if (number == avoid[i])
-      used = true;
-  return used ? noRepeat(range, avoid) : number;
-};
+  for (var i in avoid) {
+    if (number == avoid[i]) {
+      return noRepeat(range, avoid);
+    }
+  }
+  return number;
+}

-var equal = function(prediction, output) {
+function equal(prediction, output) {
  for (var i in prediction)
    if (Math.round(prediction[i]) != output[i])
      return false;
  return true;
-};
+}

-var generateRandomArray = function(size){
-    var array = [];
-    for (var j = 0; j < size; j++)
-        array.push(Math.random() + .5 | 0);
-    return array;
+function generateRandomArray(size) {
+  var array = [];
+  for (var j = 0; j < size; j++)
+    array.push(Math.random() + .5 | 0);
+  return array;
+}
+
+function calculateMse(a, b) {
+  var mse = 0;
+  for (var k in a)
+    mse += Math.pow(a[k] - b[k], 2);
+  mse /= a.length;
+
+  return mse;
+}
+
+function equalWithError(output, expected, error) {
+  return Math.abs(output - expected) <= error;
 }

 // specs

-describe('Basic Neural Network', function() {
+describe('Basic Neural Network', function () {

-  it("trains an AND gate", function() {
+  it("trains an AND gate", function () {

    var inputLayer = new Layer(2),
-        outputLayer = new Layer(1);
+      outputLayer = new Layer(1);

    inputLayer.project(outputLayer);

@@ -78,14 +96,14 @@ describe('Basic Neural Network', function() {
    var test01 = Math.round(network.activate([0, 1]));
    assert.equal(test01, 0, "[0,1] did not output 0");

-    var test10 = Math.round(network.activate([0, 1]));
+    var test10 = Math.round(network.activate([1, 0]));
    assert.equal(test10, 0, "[1,0] did not output 0");

    var test11 = Math.round(network.activate([1, 1]));
    assert.equal(test11, 1, "[1,1] did not output 1");
  });

-  it("trains an OR gate", function() {
+  it("trains an OR gate", function () {

    var inputLayer = new Layer(2),
      outputLayer = new Layer(1);
@@ -124,18 +142,17 @@ describe('Basic Neural Network', function() {
    var test01 = Math.round(network.activate([0, 1]));
    assert.equal(test01, 1, "[0,1] did not output 1");

-    var test10 = Math.round(network.activate([0, 1]));
+    var test10 = Math.round(network.activate([1, 0]));
    assert.equal(test10, 1, "[1,0] did not output 1");

    var test11 = Math.round(network.activate([1, 1]));
    assert.equal(test11, 1, "[1,1] did not output 1");
  });

-  it("trains a NOT gate", function() {
+  it("trains a NOT gate", function () {

    var inputLayer = new Layer(1),
-      outputLayer = new Layer(1),
-      network;
+      outputLayer = new Layer(1);

    inputLayer.project(outputLayer);

@@ -166,37 +183,121 @@ describe('Basic Neural Network', function() {
  });
 });

-describe("Perceptron - XOR", function() {
+describe("Perceptron - XOR", function () {

  var perceptron = new Perceptron(2, 3, 1);
  perceptron.trainer.XOR();

-  var test00 = Math.round(perceptron.activate([0, 0]));
-  it("input: [0,0] output: " + test00, function() {
-
-    assert.equal(test00, 0, "[0,0] did not output 0");
+  it("should return near-0 value on [0,0]", function () {
+    assert.isAtMost(perceptron.activate([0, 0]), .49, "[0,0] did not output 0");
  });

-  var test01 = Math.round(perceptron.activate([0, 1]));
-  it("input: [0,1] output: " + test01, function() {
-
-    assert.equal(test01, 1, "[0,1] did not output 1");
+  it("should return near-1 value on [0,1]", function () {
+    assert.isAtLeast(perceptron.activate([0, 1]), .51, "[0,1] did not output 1");
  });

-  var test10 = Math.round(perceptron.activate([1, 0]));
-  it("input: [1,0] output: " + test10, function() {
-
-    assert.equal(test10, 1, "[1,0] did not output 1");
+  it("should return near-1 value on [1,0]", function () {
+    assert.isAtLeast(perceptron.activate([1, 0]), .51, "[1,0] did not output 1");
  });

-  var test11 = Math.round(perceptron.activate([1, 1]));
-  it("input: [1,1] output: " + test11, function() {
-
-    assert.equal(test11, 0, "[1,1] did not output 0");
+  it("should return near-0 value on [1,1]", function () {
+    assert.isAtMost(perceptron.activate([1, 1]), .49, "[1,1] did not output 0");
  });
 });

-describe("LSTM - Discrete Sequence Recall", function() {
+describe("Perceptron - SIN", function () {
+  var mySin = function (x) {
+    return (Math.sin(x) + 1) / 2;
+  };
+
+  var sinNetwork = new Perceptron(1, 12, 1);
+  var trainingSet = [];
+
+  while (trainingSet.length < 800) {
+    var inputValue = Math.random() * Math.PI * 2;
+    trainingSet.push({
+      input: [inputValue],
+      output: [mySin(inputValue)]
+    });
+  }
+
+  var results = sinNetwork.trainer.train(trainingSet, {
+    iterations: 2000,
+    log: false,
+    error: 1e-6,
+    cost: Trainer.cost.MSE,
+  });
+
+  [0, .5 * Math.PI, 2]
+    .forEach(function (x) {
+      var y = mySin(x);
+      it("should return value around " + y + " when [" + x + "] is on input", function () {
+        // near scalability: abs(expected-actual) < 0.5 * 10**(-decimal)
+        // 0.5 * Math.pow(10, -.15) => 0.35397289219206896
+        assert.almostEqual(sinNetwork.activate([x])[0], y, .15);
+      });
+    });
+
+  var errorResult = results.error;
+  it("Sin error: " + errorResult, function () {
+    assert.isAtMost(errorResult, .001, "Sin error not less than or equal to desired error.");
+  });
+});
+
+describe("Perceptron - SIN - CrossValidate", function () {
+
+  var mySin = function (x) {
+    return (Math.sin(x) + 1) / 2;
+  };
+
+  var sinNetwork = new Perceptron(1, 12, 1);
+
+  var trainingSet = Array.apply(null, Array(800)).map(function () {
+    var inputValue = Math.random() * Math.PI * 2;
+    return {
+      input: [inputValue],
+      output: [mySin(inputValue)]
+    };
+  });
+
+  var results = sinNetwork.trainer.train(trainingSet, {
+    iterations: 2000,
+    log: false,
+    error: 1e-6,
+    cost: Trainer.cost.MSE,
+    crossValidate: {
+      testSize: .3,
+      testError: 1e-6
+    }
+  });
+
+  var test0 = sinNetwork.activate([0])[0];
+  var expected0 = mySin(0);
+  it("input: [0] output: " + test0 + ", expected: " + expected0, function () {
+    assert.isAtMost(Math.abs(test0 - expected0), .035, "[0] did not output " + expected0);
+  });
+
+  var test05PI = sinNetwork.activate([.5 * Math.PI])[0];
+  var expected05PI = mySin(.5 * Math.PI);
+  it("input: [0.5*Math.PI] output: " + test05PI + ", expected: " + expected05PI, function () {
+    assert.isAtMost(Math.abs(test05PI - expected05PI), .035, "[0.5*Math.PI] did not output " + expected05PI);
+  });
+
+  var test2 = sinNetwork.activate([2])[0];
+  var expected2 = mySin(2);
+  it("input: [2] output: " + test2 + ", expected: " + expected2, function () {
+    var eq = equalWithError(test2, expected2, .035);
+    assert.equal(eq, true, "[2] did not output " + expected2);
+  });
+
+  var errorResult = results.error;
+  it("CrossValidation error: " + errorResult, function () {
+    var lessThanOrEqualError = errorResult <= .001;
+    assert.equal(lessThanOrEqualError, true, "CrossValidation error not less than or equal to desired error.");
+  });
+});
+
+describe("LSTM - Discrete Sequence Recall", function () {

  var targets = [2, 4];
  var distractors = [3, 5];
@@ -234,7 +335,7 @@ describe("LSTM - Discrete Sequence Recall", function() {
    sequence.push(prompts[i]);
  }

-  var check = function(which) {
+  var check = function (which) {
    // generate input from sequence
    var input = [];
    for (j = 0; j < symbols; j++)
@@ -259,7 +360,7 @@ describe("LSTM - Discrete Sequence Recall", function() {
    };
  };

-  var value = function(array) {
+  var value = function (array) {
    var max = .5;
    var res = -1;
    for (var i in array)
@@ -270,197 +371,232 @@ describe("LSTM - Discrete Sequence Recall", function() {
    return res == -1 ? '-' : targets[res];
  };

-  it("targets: " + targets, function() {
+  it("targets: " + targets, function () {
    assert(true);
  });
-  it("distractors: " + distractors, function() {
+  it("distractors: " + distractors, function () {
    assert(true);
  });
-  it("prompts: " + prompts, function() {
+  it("prompts: " + prompts, function () {
    assert(true);
  });
-  it("length: " + length + "\n", function() {
+  it("length: " + length + "\n", function () {
    assert(true);
  });

  for (var i = 0; i < length; i++) {
    var test = check(i);
    it((i + 1) + ") input: " + sequence[i] + " output: " + value(test.prediction),
-      function() {
+      function () {
        var ok = equal(test.prediction, test.output);
        assert(ok);
      });
  }
 });

-describe("Optimized and Unoptimized Networks Equivalency", function() {
-  var optimized = new Perceptron(10,15,5);
+describe("LSTM - Timing Task", function () {
+  var network = new LSTM(2, 7, 1);
+  var result = network.trainer.timingTask({
+    log: false,
+    trainSamples: 4000,
+    testSamples: 500
+  });

-  var unoptimized = optimized.clone();
-  unoptimized.setOptimize(false);
+  it("should complete the training in less than 200 iterations", function () {
+    assert(result.train.iterations <= 200);
+  });

-  var learningRate = .5;
-  var iterations = 1000;
-
-  for (var i = 1; i <= iterations; i++)
-  {
-      //random input
-      var input = generateRandomArray(10);
-
-      // activate networks
-      var output1 = optimized.activate(input);
-      var output2 = unoptimized.activate(input);
-
-      if (i % 100 == 0)
-        it(' same output for both networks after ' + i + ' iterations', function(){
-          var diff = false;
-          for (var k in output1)
-            if (output1[k] - output2[k] != 0)
-              diff = true;
-          assert(!diff);
-        });
-
-      // random target
-      var target = generateRandomArray(5);
-
-      // propagate networks
-      optimized.propagate(learningRate, target);
-      unoptimized.propagate(learningRate, target);
-  }
+  it("should pass the test with an error smaller than 0.05", function () {
+    assert(result.test.error < .05);
+  });
 });

-describe("toJSON/fromJSON Networks Equivalency", function() {
-  var original = new Perceptron(10,15,5);
+describe("Optimized and Unoptimized Networks Equivalency", function () {

-  var exported = original.toJSON();
-  var imported = Network.fromJSON(exported);
+  var optimized;
+  var unoptimized;
+  beforeEach(function () {
+    optimized = new LSTM(2, 1, 1);
+    unoptimized = optimized.clone();
+    unoptimized.setOptimize(false);
+  });

-  var learningRate = .5;
-  var iterations = 1000;

-  for (var i = 1; i <= iterations; i++)
-  {
-      //random input
+  it('should produce the same output for both networks', function () {
+    this.timeout(30000);
+    for (var i = 0; i < 1000; i++) {
+      var input = generateRandomArray(2);
+      var target = generateRandomArray(1);
+      optimized.activate(input);
+      unoptimized.activate(input);
+      optimized.propagate(learningRate, target);
+      unoptimized.propagate(learningRate, target);
+    }
+    var mse = calculateMse(optimized.activate(input), unoptimized.activate(input));
+    assert.isAtMost(mse, 1e-9, 'output should be same for both networks after ' + i + ' iterations');
+  });
+});
+
+describe("toJSON/fromJSON Networks Equivalency", function () {
+  var original;
+  var imported;
+  beforeEach(function () {
+    original = new LSTM(10, 5, 5);
+    imported = Network.fromJSON(original.toJSON());
+  });
+
+  it('should produce the same output for both networks', function () {
+    this.timeout(30000);
+    for (var i = 0; i < 1000; i++) {
      var input = generateRandomArray(10);
-
-      // activate networks
      var output1 = original.activate(input);
      var output2 = imported.activate(input);

-      if (i % 100 == 0)
-        it(' same output for both networks after ' + i + ' iterations', function(){
-          var diff = false;
-          for (var k in output1)
-            if (output1[k] - output2[k] != 0)
-              diff = true;
-          assert(!diff);
-        });
-
-      // random target
      var target = generateRandomArray(5);

      // propagate networks
      original.propagate(learningRate, target);
      imported.propagate(learningRate, target);
-  }
+
+      assert.isAtMost(calculateMse(output1, output2), 1e-10,
+        'output should be same for both networks after ' + i + ' iterations');
+    }
+  });
 });

-describe("Cloned Networks Equivalency", function() {
-  var original = new Perceptron(10,15,5);
-  var cloned = original.clone();
+describe("Cloned Networks Equivalency", function () {
+  var original;
+  var cloned;
+  beforeEach(function () {
+    original = new LSTM(10, 5, 5);
+    cloned = Network.fromJSON(original.toJSON());
+  });

-  var learningRate = .5;
-  var iterations = 1000;
-
-  for (var i = 1; i <= iterations; i++)
-  {
-      //random input
+  it('should produce the same output for both networks', function () {
+    this.timeout(30000);
+    for (var i = 0; i < 1000; i++) {
      var input = generateRandomArray(10);
-
-      // activate networks
      var output1 = original.activate(input);
      var output2 = cloned.activate(input);

-      if (i % 100 == 0)
-        it(' same output for both networks after ' + i + ' iterations', function(){
-          var diff = false;
-          for (var k in output1)
-            if (output1[k] - output2[k] != 0)
-              diff = true;
-          assert(!diff);
-        });
-
-      // random target
      var target = generateRandomArray(5);

      // propagate networks
      original.propagate(learningRate, target);
      cloned.propagate(learningRate, target);
-  }
+
+      assert.isAtMost(calculateMse(output1, output2), 1e-10,
+        'output should be same for both networks after ' + i + ' iterations');
+    }
+  });
 });

-describe("Manual Override", function() {
+describe("Scheduled Tasks", function () {
  var perceptron = new Perceptron(2, 3, 1);

-  it('iterations ended at full 3000', function(){
+  it('should stop training at 3000 iterations', function () {
    var final_stats = perceptron.trainer.XOR({
      iterations: 3000,
      rate: 0.000001,
      error: 0.000001,
      schedule: {
-          every: 1000,
-          do: function(data) {
-            if( data.iterations == 20000){
-              return true
-            }
-          }
+        every: 1000,
+        do: function (data) {
+          return data.iterations == 20000;
        }
+      }
    });
-    assert.equal( final_stats.iterations, 3000 )
+    assert.equal(final_stats.iterations, 3000)
  });

-  it('iterations ended at 2000, not full 3000', function(){
+  it('should abort the training at 2000 iterations', function () {
    var final_stats = perceptron.trainer.XOR({
      iterations: 3000,
      rate: 0.000001,
      error: 0.000001,
      schedule: {
-          every: 1000,
-          do: function(data) {
-            if( data.iterations == 2000){
-              return true
-            }
-          }
+        every: 1000,
+        do: function (data) {
+          return data.iterations == 2000;
        }
+      }
    });
-    assert.equal( final_stats.iterations, 2000 )
+    assert.equal(final_stats.iterations, 2000)
  });

-  it('training works even when schedule() has no return value', function(){
+  it('should work even if schedule.do() returns no value', function () {
    var final_stats = perceptron.trainer.XOR({
      iterations: 3000,
      rate: 0.000001,
      error: 0.000001,
      schedule: {
-          every: 1000,
-          do: function(data) {}
-        }
+        every: 1000,
+        do: function (data) {}
+      }
    });
-    assert.equal( final_stats.iterations, 3000 )
-  });
-
-  it('using depreciated customLog still works', function(){
-    var counter = 0
-    var final_stats = perceptron.trainer.XOR({
-      iterations: 3000,
-      rate: 0.000001,
-      error: 0.000001,
-      customLog: {
-          every: 1000,
-          do: function(data) { counter++ }
-        }
-    });
-    assert.equal( counter, 3 )
+    assert.equal(final_stats.iterations, 3000)
  });

 });
+
+describe("Rate Callback Check", function () {
+  var perceptron = new Perceptron(2, 3, 1);
+
+  it('should switch rate from 0.01 to 0.005 after 1000 iterations', function () {
+    var final_stats = perceptron.trainer.XOR({
+      iterations: 2000,
+      rate: function (iterations, error) {
+        return iterations < 1000 ? 0.01 : 0.005
+      },
+      error: 0.000001,
+      schedule: {
+        every: 1,
+        do: function (data) {
+          switch (data.iterations) {
+            case 1:
+            case 500:
+            case 999:
+              assert.equal(data.rate, 0.01);
+              break;
+
+            case 1000:
+            case 1500:
+            case 2000:
+              assert.equal(data.rate, 0.005);
+              break;
+          }
+        }
+      }
+    });
+  });
+});
+
+describe("Rate Array Check", function () {
+  var perceptron = new Perceptron(2, 3, 1);
+
+  it('should switch rate from 0.01 to 0.005 after 1000 iterations', function () {
+    var final_stats = perceptron.trainer.XOR({
+      iterations: 2000,
+      rate: [0.01, 0.005],
+      error: 0.000001,
+      schedule: {
+        every: 1,
+        do: function (data) {
+          switch (data.iterations) {
+            case 1:
+            case 500:
+            case 999:
+              assert.equal(data.rate, 0.01);
+              break;
+
+            case 1000:
+            case 1500:
+            case 2000:
+              assert.equal(data.rate, 0.005);
+              break;
+          }
+        }
+      }
+    });
+  });
+});
@@ -1,22 +0,0 @@
-<!DOCTYPE html>
-<html>
-<head>
-  <title>test</title>
-  <script src="../dist/synaptic.js"></script>
-</head>
-<body>
-
-</body>
-<script type="text/javascript">
-  var Architect = synaptic.Architect;
-
-  var myPerceptron = new Architect.Perceptron(2,3,1);
-  document.body.innerHTML += JSON.stringify(myPerceptron.trainer.XOR())+'<br>'
-  document.body.innerHTML += JSON.stringify(myPerceptron.trainer.XOR())+'<br>'
-
-  if(document.body.innerHTML.length > 0){
-    document.body.innerHTML += '<br>if you see this, then synaptic works'
-  }
-
-</script>
-</html>
@@ -0,0 +1,17 @@
+var webpack = require('webpack')
+var license = require('./prebuild.js')
+module.exports = {
+  context: __dirname,
+  entry: {
+    synaptic: './src/synaptic.js',
+    'synaptic.min': './src/synaptic.js'
+  },
+  output: {
+    path: 'dist',
+    filename: '[name].js',
+  },
+  plugins: [
+    new webpack.NoErrorsPlugin(),
+    new webpack.BannerPlugin(license())
+  ]
+}
Autor	SHA1	Mensagem	Data
Vsevolod Rodionov	0b31eed0d1	Update README.md	2016-09-20 14:16:38 +03:00
Juan Cazala	a5fbefea5a	Replace juancazala.com with caza.la	2016-09-16 19:42:46 -03:00
Vsevolod Rodionov	3d8265d804	version bump for CDNJS - no actual changes see https://github.com/cazala/synaptic/issues/137	2016-09-13 12:14:35 +03:00
Vsevolod Rodionov	34c93cb718	Bringing synaptic.min.js and webpack.config that is outputting 100% identical .js and .min.js files - just a dirty fix for CDNJS that is related to https://github.com/cazala/synaptic/issues/137, we're using fn.toString() and bunch of regex-es to create a worker for now (I know, it's bad, still we already have a fix, but it is breaking some of our actually-interal, but overly-exposed APIs, so it goes to v2), and we need to either have one version minified by us or it will be minified by CDNs.	2016-09-12 01:36:48 +03:00
Juan Cazala	d3e2ea8488	Merge pull request #133 from bobalazek/master Rate error bugfix	2016-08-19 18:17:12 -03:00
Borut	4cfd5a828f	Rate error bugfix	2016-08-19 22:05:07 +02:00
Juan Cazala	24807634e8	Added slack badge	2016-08-19 12:12:56 -03:00
Juan Cazala	ebac6d8c33	Merge pull request #121 from Jabher/karma_tests bringing karma for browser testing	2016-08-03 11:26:45 -03:00
Juan Cazala	b0a9559beb	Merge pull request #120 from filet-mign0n/testreadme replaced gulp with mocha in test readme	2016-07-29 17:29:12 -03:00
Vsevolod Rodionov	5a6c487102	bringing karma for browser testing	2016-07-29 15:47:27 +03:00
Filet Mig0n	9a06d558aa	replaced gulp with mocha	2016-07-26 16:51:31 -07:00
Juan Cazala	0a544dbc9e	Replaced pre-commit with pre-push	2016-07-26 11:06:49 -03:00
Juan Cazala	3c433021e5	Bump version	2016-07-25 15:39:24 -03:00
Juan Cazala	e2240eb7b3	Added prebuild script	2016-07-25 15:38:57 -03:00
Juan Cazala	efb642c2a7	v1.0.8	2016-07-25 15:26:02 -03:00
Juan Cazala	697de49de0	v1.0.8	2016-07-25 15:25:11 -03:00
Juan Cazala	82d64f27a9	Added precommit	2016-07-25 15:23:23 -03:00
Juan Cazala	116df545fe	Merge pull request #115 from Jabher/master removing useless variables && declaring missed ones	2016-07-25 15:11:33 -03:00
Vsevolod Rodionov	e6953b5028	changing list of supported node.js versions for Travis CI in accordance with package.json manifest ("node": ">=4")	2016-07-25 17:45:06 +03:00
Vsevolod Rodionov	47745614e3	test refactor - see comment The first thing is that prev tests were relaying upon expectation of "it" function executed synchroniously. However, actual result was another: functions were called after everything was fired. So all assertions expectations like "this is result on 200th iteration" were whong as all of them were fired after all 1000 iterations ran. Another thing was default assertion library able to simple asserions. In order to simplify the debugging chai was used - it allows to write something like `assert.isAtMost(val1, val2, ...)`, and if everything fails it outputs something like "expected 0.52 to be at most 0.49", so actual BDD-style tests are used now.	2016-07-22 18:43:39 +03:00
Vsevolod Rodionov	5041da2c25	replacing assertion lib: using chai for now	2016-07-22 12:20:45 +03:00
Vsevolod Rodionov	05113ec720	dist build	2016-07-20 00:26:53 +03:00
Vsevolod Rodionov	d3044fbe2a	updating travis node.js envs	2016-07-19 15:56:34 +03:00
Vsevolod Rodionov	f6e8d79f2a	removing useless variables && declaring missed ones	2016-07-18 01:10:52 +03:00
Juan Cazala	e1ff6b86df	Merge pull request #114 from bobalazek/master Fixed rate callback and tests	2016-07-17 11:44:22 -03:00
Borut	c233227b10	Fixed rate callback and tests	2016-07-17 09:27:48 +02:00
Juan Cazala	8ff4f28075	Merge pull request #110 from bobalazek/master Rate callback	2016-07-15 10:42:04 -03:00
Borut	533d77aaea	Schould be able to set a callback the rate	2016-07-12 22:38:14 +02:00
Juan Cazala	29bd2ecf28	Delete index.html	2016-07-09 16:58:49 -03:00
Juan Cazala	8a7d82b25c	Update README.md	2016-07-09 16:55:43 -03:00
Juan Cazala	45e4cc2ab3	Delete launch.json	2016-07-09 16:52:31 -03:00
Juan Cazala	bd5062bdaa	Ignore .settings	2016-07-09 16:51:56 -03:00
Juan Cazala	0345fe648d	Fixed bundle path	2016-07-09 16:47:50 -03:00
Juan Cazala	b820493823	Merge branch 'master' of https://github.com/cazala/synaptic	2016-07-08 19:02:18 -03:00
Juan Cazala	7ee75f9984	v1.0.6	2016-07-08 19:01:04 -03:00
Juan Cazala	25e1e151a2	Replaced gulp with webpack	2016-07-08 19:00:24 -03:00
Juan Cazala	c5be6ffa3d	Merge pull request #105 from yogiben/patch-1 Add instructions to accessing examples	2016-07-07 01:50:34 -03:00
Ben Jones	50d28c328f	Update README.md	2016-07-07 00:12:35 +02:00
Juan Cazala	5524e29646	v1.0.5	2016-07-04 21:56:38 -03:00
Juan Cazala	abdce5117a	v1.0.5	2016-07-04 21:55:54 -03:00
Juan Cazala	a8d237f577	Merge pull request #102 from Cristy94/master Improve workerTrain performance	2016-07-04 21:49:12 -03:00
Cristy94	238be47c90	Remove ES6 template strings so travis build doesn't fail	2016-06-30 12:20:59 +03:00
Cristy94	4a8301d3f9	Revert accidental typo	2016-06-29 23:35:08 +03:00
Cristy94	6fc3bbf898	Improve multi-threaded training performance (#100 ) I have made the changes described in #100. After this update, training 6 networks at the same time reduced the time from over 5 minutes (I stopped it after that) to under 30 seconds, it is way, way faster. I think I didn't include the schedule call, but everything should work (including logging after X iterations).	2016-06-29 23:21:09 +03:00
Juan Cazala	8895db5052	Update README.md	2016-06-07 18:05:25 -03:00
Juan Cazala	dab9030d19	Added CDN link	2016-05-23 10:50:37 -03:00
Juan Cazala	fb0037bb0e	Merge pull request #79 from vkaracic/patch-1 Update synaptic.js	2016-03-19 12:02:39 -03:00
Vedran Karačić	74cdcc35ac	Update synaptic.js Small typo :)	2016-03-19 11:56:07 +01:00
Juan Cazala	82375f977d	Merge pull request #76 from cazala/development v1.0.4	2016-03-17 10:37:39 -03:00
Juan Cazala	b752e29788	v1.0.4	2016-03-17 10:33:57 -03:00
Juan Cazala	ab3470dd8d	Merge pull request #75 from lucasBertola/master fix bug in timingTask	2016-03-17 10:28:10 -03:00
Bertola lucas	b71201262a	fix bug in timingTask	2016-03-16 21:42:12 +01:00
Juan Cazala	e8ce3a998c	build version 1.0.3	2016-03-02 13:28:25 -03:00
Juan Cazala	afeb75eeef	build dist files	2016-03-02 13:28:12 -03:00
Juan Cazala	65dbed4936	Merge pull request #64 from funerr/corss-validation Fix options.crossValidate bug and make sin tests more lenient	2015-10-18 20:51:07 -03:00
agam360	3975fcb894	Fix options.crossValidate bug and make sin tests more lenient	2015-10-17 16:21:40 +03:00
Juan Cazala	f22842a350	Merge pull request #61 from funerr/cross-validation Cross validation	2015-10-11 16:13:57 -03:00
agam360	afa660ecde	Fix crossvalidation error bug and create its unit test	2015-10-09 03:19:43 +03:00
agam360	d9ea8fac74	Fix test	2015-10-09 03:06:36 +03:00
agam360	576e8977fd	Adding test	2015-10-09 02:31:09 +03:00
agam360	806387cae5	Fix syntax error and undefined options	2015-10-09 00:44:47 +03:00
agam360	c44c226151	Add cross-validation code	2015-10-09 00:43:06 +03:00
Juan Cazala	a5d5045784	Update bower.json	2015-09-01 11:42:41 -03:00
Juan Cazala	b2c3220c70	updated version tag	2015-09-01 11:42:24 -03:00
Juan Cazala	b1a0846e87	Merge pull request #55 from SkY3r/patch-1 >=0.10 Node Compatibility	2015-08-27 14:03:18 -03:00
SkY3r	bdefeb2853	>=0.10 Node Compatibility Error by installing on new versions of the Node: npm WARN engine synaptic@1.0.1: wanted: {"node":"^0.10.0"} (current: {"node":"0.12.7","npm":"2.13.5"})	2015-08-27 17:48:09 +02:00
Juan Cazala	7bc08646ab	Merge pull request #52 from GrantMStevens/master fixing array joinng in standalone function to prevent commas	2015-07-29 17:39:45 +02:00
Grant Stevens	a4042c4364	fixing array joinng in standalone function to prevent commas	2015-07-29 11:31:17 -04:00
Juan Cazala	4d50133633	update tag: v1.0.1	2015-07-20 03:11:42 -03:00
Juan Cazala	169bea133c	update tag: v1.0.1	2015-07-20 03:11:17 -03:00
Juan Cazala	6da278357f	fix #51	2015-07-17 13:55:02 -03:00
Juan Cazala	26dbe8ceee	added ReLU activation function	2015-07-16 10:44:04 -03:00
Juan Cazala	8e19e852a0	removed inline requires	2015-07-15 20:24:34 -03:00
Juan Cazala	06b4566eb3	fixed typo	2015-07-15 00:46:17 -03:00
Juan Cazala	96f88da732	build list files	2015-07-15 00:07:36 -03:00
Agustin Mendez	a3bd74e72a	Merge pull request #49 from Qard/real-errors Don't throw strings	2015-07-14 23:43:25 -03:00
Stephen Belanger	db2a42330e	Don't throw strings	2015-07-14 14:25:55 -07:00
Juan Cazala	e0f20561c2	updated tag	2015-07-12 23:56:45 -03:00
Juan Cazala	ec8cd5f29b	new release	2015-07-12 23:07:15 -03:00
Juan Cazala	f4eed6a396	new release	2015-07-12 23:06:42 -03:00
Juan Cazala	70b974ddcd	normalized everything to camelCase	2015-07-12 23:06:42 -03:00
Juan Cazala	892aa1b349	added new spec to the tests: timing task	2015-07-12 23:06:42 -03:00
Juan Cazala	52fe07fc95	removed browser test file	2015-07-12 23:06:42 -03:00
Juan Cazala	d4aa1d23ac	unified influences into main memory array for optimized nets	2015-07-12 23:06:41 -03:00
Juan Cazala	ed5e287a40	fixed bug: extra eligibility traces were copied when cloning	2015-07-12 23:06:41 -03:00
Juan Cazala	8be394c4c7	ignore debug file	2015-07-12 23:06:41 -03:00
Juan Cazala	26b75f9542	removed unnecesary license prepend	2015-07-12 23:06:41 -03:00
Juan Cazala	e4b1639bf4	new features: test, timingTask and scheduled tasks Trainer.cost.BINARY	2015-07-12 23:06:41 -03:00
Juan Cazala	8c4e3db35c	Merge pull request #40 from coleww/patch-1 fix toJSON comment	2015-06-21 02:25:52 -03:00
Cole Willsea	cbfe912e54	fixes toJSON comment	2015-06-20 21:47:11 -07:00
Juan Cazala	f1c5d7c9e2	Merge pull request #39 from anubisthejackle/patch-2 Update synaptic.js	2015-06-20 15:09:36 -03:00
Travis Weston	a8e7ca02df	Update synaptic.js Performance boost to noRepeat method.	2015-06-18 16:19:12 -04:00
				`@@ -0,0 +1 @@`
				`global.synaptic = require('../dist/synaptic');`
				`@@ -0,0 +1 @@`
				`global.synaptic = require('../src/synaptic');`