v1.4.1

* WIP #1031 added container dependency info

* WIP #1031 updated quickstart

.gitignore

@@ -35,3 +35,6 @@ test-tmp/
 blob-local-storage/
 src/seeds/nn/hash.txt
 src/seeds/pipeline/hash.txt
+
+notes/
+src/worker

.npmignore

@@ -38,3 +38,6 @@ src/seeds/pipeline/hash.txt
 
 # docs
 images/
+
+notes/
+src/worker

README.md

@@ -4,6 +4,8 @@
 [![Join the chat at https://gitter.im/deepforge-dev/deepforge](https://badges.gitter.im/deepforge-dev/deepforge.svg)](https://gitter.im/deepforge-dev/deepforge?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 [![Stories in Ready](https://badge.waffle.io/deepforge-dev/deepforge.png?label=ready&title=Ready)](https://waffle.io/deepforge-dev/deepforge)
 
+Using DeepForge? [Let us know what you think!](https://goo.gl/forms/2pDdCPXoUvkQhVzQ2)
+
 # DeepForge
 DeepForge is an open-source visual development environment for deep learning providing end-to-end support for creating deep learning models. This is achieved through providing the ability to design **architectures**, create training **pipelines**, and then execute these pipelines over a cluster. Using a notebook-esque api, users can get real-time feedback about the status of any of their **executions** including compare them side-by-side in real-time.
 
@@ -19,23 +21,13 @@ Additional features include:
 - Facilitates defining custom layers 
 
 ## Quick Start
-Simply run the following command to install deepforge with its dependencies:
-
+The easiest way to start deepforge is using [docker-compose](https://docs.docker.com/compose/). Using docker-compose, deepforge can be started with
 ```
-curl -o- https://raw.githubusercontent.com/dfst/deepforge/master/install.sh | bash
+wget https://raw.githubusercontent.com/deepforge-dev/deepforge/master/docker-compose.yml
+docker-compose up
 ```
 
-Or, if you already have NodeJS (v6) installed, simply run
-
-```
-npm install -g deepforge
-```
-
-Finally, start deepforge with `deepforge start`and navigate to [http://localhost:8888](http://localhost:8888) to start using DeepForge! For more, detailed instructions, check out the [wiki](https://github.com/dfst/deepforge/wiki/Installation-Guide).
-
-**Note**: running deepforge w/ `deepforge start` will also require [MongoDB](https://www.mongodb.com/download-center?jmp=nav#community) to be installed locally.
-
-Also, be sure to check out the other available features of the `deepforge` cli; it can be used to update, manage your torch installation, uninstall deepforge and run individual components!
+Finally, navigate to [http://localhost:8888](http://localhost:8888) to start using DeepForge! For more detailed instructions and other installation options, check out the [docs](http://deepforge.readthedocs.io/en/latest/deployment/overview.html).
 
 ## Additional Resources
 - [Intro to DeepForge Slides](https://docs.google.com/presentation/d/10_y5O3gHXSATfjHVLJg7dOdrz-tAXNWjlxhJ5SlA0ic/edit?usp=sharing)
@@ -46,7 +38,13 @@ Also, be sure to check out the other available features of the `deepforge` cli;
 - [Datamodel Developer Slides](https://docs.google.com/presentation/d/1hd3IyUlzW_TIPnzCnE-1pdz00Pw8WaIxYiOW_Hyog-M/edit#slide=id.p)
 
 ## Interested in contributing?
-Contributions are welcome! Either fork the project and submit some PR's or shoot me an email about getting more involved! If you have any questions, check out the [wiki](https://github.com/dfst/deepforge/wiki/) or drop me a line on the gitter!
+Contributions are welcome! There are a couple different ways to contribute to DeepForge:
+- Provide user feedback!
+    - on the [documentation](http://deepforge.readthedocs.io)
+    - on deepforge and its future development: https://goo.gl/forms/2pDdCPXoUvkQhVzQ2
+- Contribute to the project directly by submitting some PR's!
+
+If you have any questions, check out the [wiki](https://github.com/dfst/deepforge/wiki/) or drop me a line on the gitter!
 
 
 Sponsored by [Digital Reasoning](http://www.digitalreasoning.com/)

docker-compose.yml

@@ -0,0 +1,22 @@
+--- 
+services: 
+  mongo: 
+    image: mongo
+    volumes: 
+      - "$HOME/.deepforge/data:/data/db"
+  server: 
+    environment: 
+      - "MONGO_URI=mongodb://mongo:27017/deepforge"
+    image: deepforge/server
+    ports: 
+      - "8888:8888"
+    volumes: 
+      - "$HOME/.deepforge/blob:/data/blob"
+    depends_on:
+      - mongo
+  worker: 
+    command: "http://server:8888"
+    image: deepforge/worker
+    depends_on:
+      - server
+version: "2"

docs/deployment/dockerized.rst

@@ -26,11 +26,11 @@ The DeepForge server can be started with
 
 .. code-block:: bash
 
-    docker run -d -v /home/irishninja/.deepforge/blob:/data/blob \
+    docker run -d -v $HOME/.deepforge/blob:/data/blob \
     -p 8888:8888 -e MONGO_URI=mongodb://172.17.0.2:27017/deepforge \
     deepforge/server
 
-where :code:`172.17.0.2` is the ip address of the mongo container and :code:`/home/irishninja/.deepforge/blob` is the path to use for binary DeepForge data on the host. Of course, if the mongo instance is locating at a different location, :code:`MONGO_URI` can be set to this address as well. Also, the first port (:code:`8888`) can be replaced with the desired port to expose on the host.
+where :code:`172.17.0.2` is the ip address of the mongo container and :code:`$HOME/.deepforge/blob` is the path to use for binary DeepForge data on the host. Of course, if the mongo instance is locating at a different location, :code:`MONGO_URI` can be set to this address as well. Also, the first port (:code:`8888`) can be replaced with the desired port to expose on the host.
 
 Worker
 ~~~~~~

docs/deployment/native.rst

@@ -1,6 +1,30 @@
 Native Installation
 ===================
 
+Dependencies
+------------
+First, install `NodeJS <https://nodejs.org/en/>`_ (v6) and `MongoDB <https://www.mongodb.org/>`_. You may also need to install git if you haven't already.
+
+Next, you can install DeepForge using npm:
+
+.. code-block:: bash
+
+    npm install -g deepforge
+
+Now, you can check that it installed correctly:
+
+.. code-block:: bash
+
+    deepforge --version
+
+DeepForge can now be started with:
+
+.. code-block:: bash
+
+    deepforge start
+
+However, the first time DeepForge is started, it will make sure that the deep learning framework is installed (if it isn't found on the host system). This may require you to start DeepForge a couple times; the first time it starts it will install Torch7 and require a terminal restart to update a couple environment variables (like `PATH`). The second time it starts it will install additional torch packages but will not require a terminal restart. Finally, DeepForge will start with all the required dependencies.
+
 Database
 ~~~~~~~~
 Download and install MongoDB from the `website <https://www.mongodb.org/>`_. If you are planning on running MongoDB locally on the same machine as DeepForge, simply start `mongod` and continue to setting up DeepForge.

docs/getting_started/hello_cifar.rst

@@ -28,7 +28,7 @@ In this example, we have three main pipelines: :code:`download-normalize`, :code
 
 First, we will have to retrieve and prepare the data by running the :code:`download-normalize` pipeline. This can be done by opening the given pipeline then selecting the `Execute Pipeline` option from the action button in the lower right. As soon as that pipeline finishes, we can now use this data to train a neural network.
 
-Next, we can open the :code:`train` pipeline. Before we execute the pipeline we have to set the input trainning data that we will be using. This is done by selecting the :code:`Input` operation then clicking the value for the :code:`artifact` field. This will provide all the possible options for the input data; for this example, we will want to select the "trainingdata" artifact. After setting the input, we can click on the :code:`train` operation to inspect the hyperparameters we are using and the architecture we are training. Selecting the :code:`Output` operation will allow you to change the name of the resulting artifact of this operation (in this case, a trained model). Finally, we can execute this pipeline like before to train the model.
+Next, we can open the :code:`train` pipeline. Before we execute the pipeline we have to set the input training data that we will be using. This is done by selecting the :code:`Input` operation then clicking the value for the :code:`artifact` field. This will provide all the possible options for the input data; for this example, we will want to select the "trainingdata" artifact. After setting the input, we can click on the :code:`train` operation to inspect the hyperparameters we are using and the architecture we are training. Selecting the :code:`Output` operation will allow you to change the name of the resulting artifact of this operation (in this case, a trained model). Finally, we can execute this pipeline like before to train the model.
 
 .. figure:: select_train_data.png
     :align: center

docs/getting_started/quick_start.rst

@@ -1,29 +1,19 @@
 Quick Start
 ===========
-Before we can start with the examples, we will first install DeepForge locally.
+The easiest way to get started quickly with DeepForge is using docker-compose. First, install `docker <https://docs.docker.com/engine/installation/>`_ and `docker-compose <https://docs.docker.com/compose/install/>`_.
 
-Dependencies
-------------
-First, install `NodeJS <https://nodejs.org/en/>`_ (v6) and `MongoDB <https://www.mongodb.org/>`_. You may also need to install git if you haven't already.
-
-Next, you can install DeepForge using npm:
+Next, download the docker-compose file for DeepForge:
 
 .. code-block:: bash
 
-    npm install -g deepforge
+    wget https://raw.githubusercontent.com/deepforge-dev/deepforge/master/docker-compose.yml
 
-Now, you can check that it installed correctly:
+Then start DeepForge using docker-compose:
 
 .. code-block:: bash
 
-    deepforge --version
+    docker-compose up
 
-DeepForge can now be started with:
-
-.. code-block:: bash
-
-    deepforge start
-
-However, the first time DeepForge is started, it will make sure that the deep learning framework is installed (if it isn't found on the host system). This may require you to start DeepForge a couple times; the first time it starts it will install Torch7 and require a terminal restart to update a couple environment variables (like `PATH`). The second time it starts it will install additional torch packages but will not require a terminal restart. Finally, DeepForge will start with all the required dependencies.
+and now DeepForge can be used by opening a browser to `http://localhost:8888 <http://localhost:8888>`_!
 
 For detailed instructions about deployment installations, check out our `deployment installation instructions <getting_started/configuration.rst>`_

package.json

@@ -17,7 +17,7 @@
     "watch-test": "nodemon --exec 'mocha --recursive test'",
     "build-nn": "node ./utils/nn-parser.js"
   },
-  "version": "1.3.0",
+  "version": "1.4.1",
   "dependencies": {
     "commander": "^2.9.0",
     "dotenv": "^2.0.0",

src/decorators/ContainerLayerDecorator/EasyDAG/NestedLayer.js

@@ -42,8 +42,8 @@ define([
             embedded: true,
             widget: this.widget
         });
-        this.control._onUnload = () => {
-            ArchEditor.prototype._onUnload.apply(this.control, arguments);
+        this.control._onUnload = id => {
+            ArchEditor.prototype._onUnload.call(this.control, id);
             // If it was the last node, remove it
             var node = this.control._client.getNode(this.id);
             if (node.getChildrenIds().length === 0) {

src/visualizers/widgets/TextEditor/TextEditorWidget.js

@@ -5,16 +5,24 @@ define([
     'ace/ace',
     'underscore',
     './completer',
+    'js/Utils/ComponentSettings',
+    'jquery-contextMenu',
     'css!./styles/TextEditorWidget.css'
 ], function (
     ace,
     _,
-    Completer
+    Completer,
+    ComponentSettings
 ) {
     'use strict';
 
     var TextEditorWidget,
-        WIDGET_CLASS = 'text-editor';
+        WIDGET_CLASS = 'text-editor',
+        DEFAULT_SETTINGS = {
+            keybindings: 'default',
+            theme: 'solarized_dark',
+            fontSize: 12
+        };
 
     TextEditorWidget = function (logger, container) {
         this._logger = logger.fork('Widget');
@@ -27,9 +35,13 @@ define([
 
         this.readOnly = this.readOnly || false;
         this.editor = ace.edit(this.$editor[0]);
+        this._initialize();
 
         // Get the config from component settings for themes
         this.editor.getSession().setOptions(this.getSessionOptions());
+        var handler = this.editorSettings.keybindings;
+        this.editor.setKeyboardHandler(handler === 'default' ?
+            null : 'ace/keyboard/' + handler);
         this.addExtensions();
         this.editor.$blockScrolling = Infinity;
         this.DELAY = 750;
@@ -47,7 +59,6 @@ define([
         this.setReadOnly(this.readOnly);
         this.currentHeader = '';
         this.activeNode = null;
-        this._initialize();
 
         this._logger.debug('ctor finished');
     };
@@ -68,7 +79,8 @@ define([
         return {
             enableBasicAutocompletion: true,
             enableLiveAutocompletion: true,
-            fontSize: '12pt'
+            theme: 'ace/theme/' + this.editorSettings.theme,
+            fontSize: this.editorSettings.fontSize + 'pt'
         };
     };
 
@@ -83,6 +95,124 @@ define([
     TextEditorWidget.prototype._initialize = function () {
         // set widget class
         this._el.addClass(WIDGET_CLASS);
+
+        // Add context menu
+        $.contextMenu('destroy', '.' + WIDGET_CLASS);
+        $.contextMenu({
+            selector: '.' + WIDGET_CLASS,
+            build: $trigger => {
+                return {
+                    items: this.getMenuItemsFor($trigger)
+                };
+            }
+        });
+
+        // Create the editor settings
+        this.editorSettings = _.extend({}, DEFAULT_SETTINGS),
+        ComponentSettings.resolveWithWebGMEGlobal(
+            this.editorSettings,
+            this.getComponentId()
+        );
+    };
+
+    TextEditorWidget.prototype.getComponentId = function () {
+        return 'TextEditor';
+    };
+
+    TextEditorWidget.prototype.getMenuItemsFor = function () {
+        var fontSizes = [8, 10, 11, 12, 14],
+            themes = [
+                'Solarized Light',
+                'Solarized Dark',
+                'Twilight',
+                'Tomorrow Night',
+                'Eclipse',
+                'Monokai'
+            ],
+            keybindings = [
+                'default',
+                'vim',
+                'emacs'
+            ],
+            menuItems = {
+                setKeybindings: {
+                    name: 'Keybindings...',
+                    items: {}
+                },
+                setFontSize: {
+                    name: 'Font Size...',
+                    items: {}
+                },
+                setTheme: {
+                    name: 'Theme...',
+                    items: {}
+                }
+            };
+
+        fontSizes.forEach(fontSize => {
+            var name = fontSize + ' pt',
+                isSet = fontSize === this.editorSettings.fontSize;
+
+            if (isSet) {
+                name = '<span style="font-weight: bold">' + name + '</span>';
+            }
+
+            menuItems.setFontSize.items['font' + fontSize] = {
+                name: name,
+                isHtmlName: isSet,
+                callback: () => {
+                    this.editorSettings.fontSize = fontSize;
+                    this.editor.setOptions(this.getEditorOptions());
+                    this.onUpdateEditorSettings();
+                }
+            };
+        });
+
+        themes.forEach(name => {
+            var theme = name.toLowerCase().replace(/ /g, '_'),
+                isSet = theme === this.editorSettings.theme;
+
+            if (isSet) {
+                name = '<span style="font-weight: bold">' + name + '</span>';
+            }
+
+            menuItems.setTheme.items[theme] = {
+                name: name,
+                isHtmlName: isSet,
+                callback: () => {
+                    this.editorSettings.theme = theme;
+                    this.editor.setOptions(this.getEditorOptions());
+                    this.onUpdateEditorSettings();
+                }
+            };
+        });
+
+        keybindings.forEach(name => {
+            var handler = name.toLowerCase().replace(/ /g, '_'),
+                isSet = handler === this.editorSettings.keybindings;
+
+            if (isSet) {
+                name = '<span style="font-weight: bold">' + name + '</span>';
+            }
+
+            menuItems.setKeybindings.items[handler] = {
+                name: name,
+                isHtmlName: isSet,
+                callback: () => {
+                    this.editorSettings.keybindings = handler;
+                    this.editor.setKeyboardHandler(handler === 'default' ?
+                        null : 'ace/keyboard/' + handler);
+                    this.onUpdateEditorSettings();
+                }
+            };
+        });
+
+        return menuItems;
+    };
+
+    TextEditorWidget.prototype.onUpdateEditorSettings = function () {
+        ComponentSettings.overwriteComponentSettings(this.getComponentId(), this.editorSettings,
+            err => err && this._logger.error(`Could not save editor settings: ${err}`));
     };
 
     TextEditorWidget.prototype.onWidgetContainerResize = function () {
@@ -147,6 +277,7 @@ define([
     /* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
     TextEditorWidget.prototype.destroy = function () {
         this.editor.destroy();
+        $.contextMenu('destroy', '.' + WIDGET_CLASS);
     };
 
     TextEditorWidget.prototype.onActivate = function () {