Comparar commits

..

171 Commits

Autor SHA1 Mensagem Data
Brian Broll 5135ccef09 v0.9.0 2016-07-18 08:39:49 -05:00
Brian Broll 6314e00a8d Fixed the attribute shown in deserialize viz. Fixes #512 (#517) 2016-07-17 21:10:22 -05:00
Brian Broll 779429e24d Fixed obj changed pass thru in main view. Fixes #513 (#514) 2016-07-17 20:34:15 -05:00
Brian Broll a7f4eac09d Updated arch editor tooltip init. Fixes #515 (#516) 2016-07-17 20:34:01 -05:00
Brian Broll 7c645e9b23 Added unused op input warning. Fixes #499 (#511)
WIP #499 Added unused input detection

WIP #499. Added input highlight updating on code change

WIP #499 Added warning tooltip and fixed removal

WIP #499 Fixed warning location on position change

WIP #499 Removed unnecessary timeout

WIP #499 Added sophisticated usage checking

WIP #499 Removed code climate issues
2016-07-16 22:42:14 -05:00
Brian Broll fc3e14644f Updated the selObjChanged to update pipeline list. Fixes #507 (#508)
WIP #507 set webgme-autoviz to track master
2016-07-16 15:37:59 -05:00
Brian Broll fd48c1b480 Removed panel padding. Fixes #502 (#506)
WIP #502 Removed padding from the tiling viz's panel
2016-07-16 14:14:11 -05:00
Brian Broll 8928d0f105 Removed old 'code' attr. Fixes #93 (#505) 2016-07-16 13:53:11 -05:00
Brian Broll 6857da5ac7 Updated webgme-autoviz. Fixes #503 (#504) 2016-07-16 13:40:01 -05:00
Brian Broll fee584cfde Added lenet arch to cifar10 example. Fixes #66 (#501) 2016-07-15 20:13:24 -05:00
Brian Broll 45fcd81739 Added tracking EOF for log viewer. Fixes #304 (#500)
WIP #304 Added scroll to bottom

WIP #304 Added eof detection
2016-07-15 17:48:14 -05:00
Brian Broll d831f5e032 Added ubuntu support to install script. Fixes #395 (#498)
WIP #395 Added ubuntu/arch detection

WIP #395 added manual nvm loading

WIP #395 Added source post-installation instructions

WIP #395 Added ubuntu/arch detection
2016-07-15 16:06:18 -05:00
Brian Broll 65e938f489 Created better execution fork names. Fixes #494 (#496)
Improved execution fork name. Fixes #494
2016-07-15 10:25:33 -05:00
Brian Broll b9b5cea5a1 Added exec fork notification. Fixes #345 (#495) 2016-07-15 10:08:26 -05:00
Brian Broll 6c639376d1 Added attributes to condense for width calc. Fixes #492 (#493) 2016-07-15 09:27:03 -05:00
Brian Broll 58ed3048f8 Updated OperationDecorator transitions. Fixes #490 (#491) 2016-07-15 07:41:00 -05:00
Brian Broll f64492061d Added operation attribute support. Fixes #487 (#489)
WIP #487 Added attr edit/del fn-ality

WIP #487 Added attribute creation

WIP #487 Fixed invalid name list

WIP #487 Fixed code climate issue

WIP #487 Fixed operation only behavoirs

WIP #487 Removed 'code' attribute and added number check

WIP #487 Added scientific notation detection

WIP #487 Updated seeds
2016-07-14 12:19:04 -05:00
Brian Broll 552e71687d Added editors to preload by autoviz. Fixes #479 (#486)
WIP #479 Updated autoviz version
2016-07-13 09:37:29 -05:00
Brian Broll 448de23945 Check if instance exists before invoking ser method. Fixes #484 (#485) 2016-07-11 13:53:25 -05:00
Brian Broll dbd00e5946 Added filter for 'return to' action btns. Fixes #482 (#483) 2016-07-11 13:38:38 -05:00
Brian Broll 4d0b4ae017 Added primitive renaming support (by editing comment). Fixes #444 (#480)
WIP #444 Naming support for SerializeEditor

WIP #444 Added renaming functionality for the deserialize editor

WIP #444 Fixed code climate issues
2016-07-11 13:03:20 -05:00
Brian Broll d6c201cce6 Retrieve the deser fn from source node. Fixes #245 (#478) 2016-07-11 12:12:08 -05:00
Brian Broll b8bc3af524 Escaped quoted operation name in log. Fixes #470 (#475) 2016-07-11 11:48:54 -05:00
Brian Broll 72de1d44da Set inputs/outputs for all nodes in pipeline editor. Fixes #411 (#473) 2016-07-11 10:51:02 -05:00
Brian Broll da6f186b52 Added string, boolean types. Fixes #471 (#474) 2016-07-11 10:50:57 -05:00
Brian Broll 5f1c50beb6 Removed territory on text editor destory. Fixes #409 (#472) 2016-07-11 10:50:52 -05:00
Brian Broll 3a79afbc41 v0.8.0 2016-07-11 10:30:22 -05:00
Brian Broll ba210e4b27 Updated icons. Fixes #406 (#469)
WIP #406 Updated some icons

WIP #406 Updated the terminal icon

WIP #406 Updated operation interface icons

WIP #406 terminal -> monitor icon

WIP #406 Fixed code climate issue
2016-07-09 17:15:44 -05:00
Brian Broll bdb21a5262 Set title to regular case. Fixes #467 (#468) 2016-07-09 16:05:30 -05:00
Brian Broll 4b6471efbc Updated buttons and methods for webgme-easydag update. Fixes #465 (#466)
WIP #465 Updated onAddButtonClicked

WIP #465 Updated callbacks for webgme-easydag update

WIP #465 Fixing code climate issue
2016-07-09 15:42:04 -05:00
JimnyCricket beeb1deb3c Merge pull request #457 from dfst/456-simple-cifar
Split cifar10 example into 4 pipelines. Fixes #456
2016-07-08 17:37:19 -05:00
Brian Broll 0bc6846c32 Optimized the arch editor layer selection. Fixes #78 (#463)
WIP #78 ~3x speedup in arch editor

WIP #78 Removed unused var
2016-07-08 10:00:47 -06:00
Brian Broll 8a40cb95fb Updated and configured breadcrumbheader. Fixes #441 (#462) 2016-07-08 07:29:45 -06:00
Brian Broll 92ea4b2ff5 Removed ptrs from op desc in OpIntEditor. Fixes #433 (#461) 2016-07-07 17:48:56 -06:00
Brian Broll e0a83b4d6c Added 'print', 'SGD' ops. Fixes #368 (#460) 2016-07-07 17:41:58 -06:00
Brian Broll 49268b9554 Fixed arch update on main screen. Fixes #458 (#459) 2016-07-07 17:41:51 -06:00
Brian Broll 65164bcb76 Merge branch 'master' into 456-simple-cifar 2016-07-07 17:31:43 -05:00
Brian Broll 1afcaf9238 Split cifar10 example into 4 pipelines. Fixes #456 2016-07-07 17:30:34 -05:00
Brian Broll 92021a06ba Decreased cifar10 max iterations. Fixes #424 (#455) 2016-07-07 15:48:18 -06:00
Brian Broll be1e4fa388 Added 'Data' to op input options. Fixes #376 (#454)
WIP #376 Removed 'Data' from output options
2016-07-07 15:17:27 -06:00
JimnyCricket 09505c9989 Clears jobs on restart of server Fixes #452 (#453) 2016-07-07 14:57:34 -06:00
Brian Broll b96f2a845a Added criterion reference support. Fixes #369 (#450)
WIP #369 Removed criterion filter

WIP #396 Added criterion layers to nn seed

WIP #369 Added GenerateCriterion plugin

WIP #369 Added GenerateCriterion to criterion meta

WIP #369 Filtered criterion from the ArchEditor

WIP Removed old css

WIP #369 filtered criterion types

WIP #369. Fixed criterion pointer assignment

WIP #369 Fixed criterion execution problems

WIP #369 Updated seeds

WIP #369 Fixed failing test
2016-07-07 11:54:09 -06:00
Brian Broll 14c0af86c9 Added import torch action to arch editor. Fixes #447 (#449) 2016-07-07 10:29:13 -06:00
Brian Broll 56a0b788ee Added editable title for architecture. Fixes #445 (#448)
WIP #445 Refactoring the panel title renaming

WIP #445 Made arch editor renameable

WIP #445 Fixed capitalization for consistency
2016-07-07 10:28:54 -06:00
Brian Broll 873cbc2145 Updated main view to have nav bar and pipeline index. Fixes #440 (#446)
WIP #440 Added MainView

WIP #440 Removed unneeded files

WIP #440 Added basic left bar

WIP #440 Added root node actions

WIP #440 Embedded PipelineIndex

WIP #440 Removed unused js

WIP #440 Removed setTimeout

WIP #440 Fixed some layout issues. css reduction

WIP #440 Added glyphicon

WIP #440 Fixed menu control icon placement, size, color

WIP

WIP #440 Added opening/closing of the nav bar

WIP #440 Added grey background-color

WIP #440 Fixed hide on small screens

WIP #440. Added action buttons

WIP #440 Fixed arch-content list

WIP #440 Removed bullet points

WIP #440 Fixed resize positioning

WIP #440 Increased font size for headers

WIP #440 italic actions

WIP #440 Added some structure for the artifacts

WIP #440 Added download artifact link

WIP #440 Fixed lingering territory rules

WIP #440 removed unused lib files

WIP #440 Fixed code climate issues
2016-07-07 09:21:35 -06:00
Brian Broll 3e3326688c Added meaningful error if retrieving nil val from layer. Fixes #386 (#442)
WIP #386 Added googlenet-setters for testing error messages

WIP #386 Commented out tacking in extra values

WIP #386 Ignore "nil gets" if in "set" operation
2016-07-01 13:14:06 -05:00
Brian Broll 6e32394e01 Added pipeline index viz. Fixes #429 (#438)
WIP #429 Added basic boilerplate for the pipeline index viz

WIP #429 Removed the title

WIP #429 Fixed columns (and text)

WIP #429 Added open, delete actions

WIP #429 Added double click to edit pipeline name

WIP #429 Added check for empty, same name

WIP #429 Added comment for thing to add next

WIP #429 Added some thumbnail support and embedded support

WIP #429 Fixed the thumbnail

WIP #429 thumbnail updates

WIP #429 Set background color and fixed updateNode bug

WIP #429 Fixed update thumbnail deletion

WIP #429 Added preview click-to-open

WIP #429 Added empty message

WIP #429 slight refactor

WIP #429 Added executions text (better phrasing)

WIP #429 Removed toolbar items

WIP #429 Fixed remove undefined bug

WIP #429 Updated seeds

WIP #429 Added pipeline index css to ignore

WIP #429 Fixed code climate issues

WIP #429 Added csslint ignore file

WIP #429 Fixed code climate issues

WIP #429 Added csslint ignore block for materialize

WIP #429 Added thumbnails to cifar10 example
2016-06-30 15:33:05 -05:00
Brian Broll afed957af8 Set artifact loader to remove data handle on "unset". Fixes #437 (#439) 2016-06-30 15:24:42 -05:00
Brian Broll 4c100ac001 Added support for creating primitive types from op editor. Fixes #432 (#435)
WIP #432 Added basic support for primitive/complex coloring and creation

WIP #432 Added tabs
2016-06-30 14:46:06 -05:00
Brian Broll 29724551f0 Updated reference selection btns in op editor. Fixes #434 (#436) 2016-06-29 14:50:18 -05:00
Brian Broll 1f5c8d7423 Updated ImportArtifact result checking. Fixes #427 (#431) 2016-06-29 13:24:18 -05:00
Brian Broll a613a1e8f7 Added artifact upload from ArtifactLoader. Fixes #422 (#428)
WIP #422 Added upload artifact icon

WIP #422 Added upload artifact functionality
2016-06-29 13:10:09 -05:00
JimnyCricket 7fce6e98dd Merge pull request #426 from dfst/425-port-must-be-num
Casted PORT to num. Fixes #425
2016-06-29 11:04:09 -07:00
Brian Broll ea572e8f6c Casted PORT to num. Fixes #425 2016-06-29 12:26:24 -05:00
Brian Broll 36290d8dc7 Removed the data type card. Fixes #420 (#421) 2016-06-29 09:32:50 -05:00
Brian Broll dadc09e0e1 Added delete, return-to-op fab actions for Primitive editing. Fixes #418 (#419) 2016-06-29 09:07:17 -05:00
Brian Broll be3d278d13 Added GoToBase btn for data in op editor. Fixes #414 (#417) 2016-06-29 09:00:22 -05:00
Brian Broll 32a96fbf2b Added class def -> op editor pivots and del btn. Fixes #413 (#416)
WIP #413. Fixed code climate issue
2016-06-29 08:51:38 -05:00
Brian Broll ebe391c948 Added 'Create Class' node in OperationEditor. Fixes #171 (#415)
WIP #171 Adding support for new classes

WIP #171. Fixed "New Class" tooltip

WIP #171 Fixed eslint error
2016-06-29 08:41:15 -05:00
Brian Broll e30ebf3a45 Added os not recognized message. Fixes #404 (#405)
WIP #404 Added "install mongo" message at end

WIP #404 Fixed message when mongo installed
2016-06-28 15:43:37 -05:00
Brian Broll 9e0a780ed2 Updated new op name to conform to new layer name. Fixes #399 (#403) 2016-06-28 15:17:58 -05:00
Brian Broll 841637e804 Updated plugin names and icons. Fixes #401 (#402) 2016-06-28 15:01:44 -05:00
Brian Broll 75241262c4 Added delete custom layer def. Fixes #389 (#400)
WIP #389 code climate fix
2016-06-28 14:37:40 -05:00
Brian Broll b39ac022b6 Added operation deletion button. Fixes #390 (#398)
WIP #390 Added delete button (non-func)

WIP #390 Added deletion func for deleting current node
2016-06-28 14:20:05 -05:00
Brian Broll 8486b86a05 Added pivot back to last arch from custom layer. Fixes #388 (#396)
WIP #388 Refactored op -> pipeline pivoting
2016-06-28 13:53:36 -05:00
Brian Broll 322af8a4f5 Removed 'MyLayers' card. Fixes #391 (#394) 2016-06-28 13:39:49 -05:00
Brian Broll 5e1930f096 Added tabs to initial layer dialog. Fixes #392 (#393) 2016-06-28 13:09:28 -05:00
Brian Broll c0f1348a14 Added sponsor info and removed caffe support comment 2016-06-28 13:06:35 -05:00
Brian Broll 383f5f9fa0 Added (tabbed) custom layer creation from arch editor. Fixes #385 (#387)
WIP #385 Added node icons to arch creator.

WIP #385 Updated for new AddNodeDialog.prompt interface

WIP #385 Added tabs to layers and create layer support

WIP #385 Added code climate ignore dup (non) issue

WIP Added codeclimate, csslint configs
2016-06-28 10:50:40 -05:00
Brian Broll 5cf592e103 removed old 'demo' seed. Fixes #365 (#384) 2016-06-28 07:46:26 -05:00
Brian Broll a14e750b6f Made pipeline title editable. Fixes #375 (#383) 2016-06-27 16:12:25 -05:00
Brian Broll 73e165197f Added editable operation title & better header comment. Fixes #380 (#381)
WIP #380 Added editable title

WIP #380 Updated the operation header comment

WIP #380 Fixed code climate warning
2016-06-27 14:26:59 -05:00
Brian Broll 9dd0815625 Auto-remove dangling connections in pipelines. Fixes #377 (#382) 2016-06-27 14:26:51 -05:00
Brian Broll 8e3ac1d203 Removed Ops, Execs cards from root view. Fixes #373 (#379) 2016-06-27 13:45:32 -05:00
Brian Broll 65f0902dd3 v0.7.0 2016-06-27 11:52:03 -05:00
Brian Broll 825209970c Added class creation. Fixes #334 (#378)
WIP #334 Fixed programmatic changes triggeringd save

WIP #334 Added support for single panel

WIP #334 Added ClassEditor

WIP #334 Added boilerplate class code

WIP #334 Updated creation buttons

WIP #334 Updated pipeline execution for complex data types

WIP #334 Fixed problem in default template

WIP #334 Updated default code template

WIP #334 Updated class name from src return value

WIP #334 Removed Primitive, Complex data types from opint editor

WIP #334 set isAbstract if needed on data type creation

WIP #334 updated seeds

WIP #334 Updated cifar example
2016-06-27 11:43:16 -05:00
Brian Broll 2453aa8f05 Added work in progress disclaimer 2016-06-27 08:01:46 -05:00
Brian Broll 9fca9783a0 Moved attr check to after node id validation. Fixes #366 (#367)
WIP #366 Fixed activeNode check in TextEditor
2016-06-25 13:46:57 -05:00
Brian Broll 62616d28db Added better failed pipeline handling. Fixes #356 (#362) 2016-06-24 16:20:02 -05:00
Brian Broll db03f3f361 Added check for null eid and cleaned up territory ui. Fixes #357 (#361)
WIP #357 Removing UI if no originId

WIP #357 Checked for null event id
2016-06-24 16:16:45 -05:00
Brian Broll 68e4e06f3b Removing UI before creating new one - prevents race cond. Fixes #359 (#360) 2016-06-24 15:35:44 -05:00
Brian Broll 4e03997af6 Added execution status updating and visualization. Fixes #354 (#358)
WIP #354 Added execution status updating and visualization

WIP #354 updated seeds
2016-06-24 15:35:25 -05:00
Brian Broll 9357503b49 Added execution viewer in pipeline editor. Fixes #353 (#355)
* Added execution viewer in pipeline editor. Fixes #353

WIP #353 Added basic execution viewer

WIP #353 Added delete icon for executions

WIP #353 Fixed width w/ long names

WIP #353 Fixed execution tab scrolling

* WIP #353 Fixed code climate issues
2016-06-24 12:55:57 -05:00
Brian Broll 24b7471d28 Added test for renaming op input/outputs. Fixes #351 (#352)
WIP #352 Added valid name check for references
2016-06-24 08:23:38 -05:00
Brian Broll 9e16aba11b 347 scroll support node prompter (#350)
* Added expandable operation creator from fab. Fixes #342

WIP UX changes...

Added expandable operation creator from fab. Fixes #342

WIP #342 Added nodes to the node prompt

WIP #342 Fixed horizontal positioning

WIP #342 Flexible sized dialog

WIP #342 Added rounded corners

WIP #342 Added padding to cover btn shadow

WIP #342 Hide tooltip and close fab on open

WIP #342 Fixed not closing on "click-and-run"

WIP #342 Re-added cards for RootViz

* Added support for creating new operations from pipeline view. Fixes #243

WIP #243 new operation from pipeline view

WIP #243 Added DeepForge global

WIP #243 new operation support from pipeline view complete

* Added scroll support. Fixes #347

* WIP #347 Fixed code climate TODO
2016-06-24 06:03:52 -05:00
Brian Broll 60f7cba7e7 243 new op from pipeline (#349)
* Added expandable operation creator from fab. Fixes #342

WIP UX changes...

Added expandable operation creator from fab. Fixes #342

WIP #342 Added nodes to the node prompt

WIP #342 Fixed horizontal positioning

WIP #342 Flexible sized dialog

WIP #342 Added rounded corners

WIP #342 Added padding to cover btn shadow

WIP #342 Hide tooltip and close fab on open

WIP #342 Fixed not closing on "click-and-run"

WIP #342 Re-added cards for RootViz

* Added support for creating new operations from pipeline view. Fixes #243

WIP #243 new operation from pipeline view

WIP #243 Added DeepForge global

WIP #243 new operation support from pipeline view complete
2016-06-24 05:57:11 -05:00
Brian Broll 7ec0c98d5c Added expandable operation creator from fab. Fixes #342 (#348)
* Added expandable operation creator from fab. Fixes #342

WIP UX changes...

Added expandable operation creator from fab. Fixes #342

WIP #342 Added nodes to the node prompt

WIP #342 Fixed horizontal positioning

WIP #342 Flexible sized dialog

WIP #342 Added rounded corners

WIP #342 Added padding to cover btn shadow

WIP #342 Hide tooltip and close fab on open

WIP #342 Fixed not closing on "click-and-run"

WIP #342 Re-added cards for RootViz

* WIP #342 Fixed code climate issues
2016-06-24 05:49:33 -05:00
JimnyCricket 03339bb667 Merge pull request #344 from dfst/343-local-ops-not-recog
Fixed snapshot generation wrt local ops. Fixes #343
2016-06-23 18:04:05 -05:00
Brian Broll a77ca29bbb WIP #343 Fixed code climatee issues 2016-06-23 10:10:45 -05:00
Brian Broll 7fe29c705c Fixed snapshot generation wrt local ops. Fixes #343 2016-06-23 10:05:03 -05:00
Brian Broll 2d20746808 Added support for navigating to original pipeline from exec. Fixes #340 (#341)
* Added support for navigating to original pipeline from exec. Fixes #340

WIP #340 Solved using entirely attributes

WIP #340 Updated to use pointers

WIP #340 Updated seeds

WIP #340 fixed minor bug in dev view

* WIP #340 Fixed code climate issues

* WIP #340 Fixed snapshot ui bug
2016-06-22 11:00:04 -05:00
Brian Broll d1b43bf50c Added support for (color coded) 'queued' jobs. Fixes #330 (#339)
WIP #330 Added support for 'queued' jobs

WIP #330 updated seeds

WIP #330 lightened grey color for queued jobs
2016-06-22 08:23:23 -05:00
Brian Broll 80f87b1068 Added execution snapshot support. Fixes #337 (#338)
WIP #337 Added code snapshot support

WIP #377 Adding support for data node independent snapshots

WIP #337 Added visual cue for execution snapshots

WIP #337 Added unique execution naming and making labels

WIP #337 'tag' -> 'tagname'

WIP #337 Updated seeds
2016-06-22 08:01:07 -05:00
JimnyCricket a3ff6cf40d Added support for variable arguments in nn importer. Fixes #328 (#329)
WIP #328 Renamed arg to 'params'. Fixed eslint issues

WIP #328 Updated nn layers.json

WIP #328 Added ReLU6 layer

WIP #328 Updated seeds
2016-06-21 14:47:24 -05:00
JimnyCricket ec8fcb25f0 Completed CIFAR-10 Demo seed. Fixes #77 (#331)
WIP #77 Removed old options values and updated some logs

WIP #77 Removed ImportTorch reg value; updated pipeline lib
2016-06-21 14:26:52 -05:00
Brian Broll d9899af233 Merge branch '155-import-torch-namespace' 2016-06-21 09:19:09 -05:00
Brian Broll 823e038edc Added simplified import arch button. Fixes #155
WIP #155 Added plugin message on successful import

WIP #155. Added Import Torch Architecture button

WIP #155 Removed ImportTorch from plugin registry

WIP #155 Added eslint global
2016-06-21 08:48:16 -05:00
Brian Broll 7eb2c1a404 v0.6.0 2016-06-20 16:22:47 -05:00
JimnyCricket 69cfc4dd77 Merge pull request #325 from dfst/324-artifact-loader-returns-nil
Updated downcast data setting. Fixes #324
2016-06-20 15:49:56 -04:00
Brian Broll e88500df33 Updated pipeline buttons. Fixes #326 (#327) 2016-06-20 12:01:22 -05:00
Brian Broll 64ae1da03e Updated downcast data setting. Fixes #324
WIP #324. Fixed data reset
2016-06-20 11:39:57 -05:00
Brian Broll e24bf3171d Added simple install script. Fixes #278 (#323)
WIP #278 added cloning deepforge repo

WIP #278 updated readme and added more installation logging

WIP #278 added FIXME when merging

WIP #278 Added curl to readme

WIP #278 Fixed var scope

WIP #278 Updated readme
2016-06-20 11:07:56 -05:00
JimnyCricket f301e9312f Merge pull request #320 from dfst/306-terminal-ansi-codes
Added ansi color support. Fixes #306
2016-06-20 10:51:19 -04:00
Brian Broll 0b95219c2a Removed extra seeds. Fixes #317 (#322) 2016-06-20 09:45:00 -05:00
Brian Broll b05f96e77a Fixed promise chaining issue in ExecutePipeline. Fixes #310 (#321) 2016-06-20 09:27:50 -05:00
Brian Broll 063aac8730 Added ansi color support. Fixes #306
WIP #306 Added basic ansi color support (flicker-y)

WIP #306 Fixed resize color update

WIP #306 Overrode the ace renderer to add support for ansi

WIP #306 Removed old TODO
2016-06-20 09:13:15 -05:00
Brian Broll edeb783547 Custom Layer Support (#315)
* Added basic custom layer editor. Fixes #1

* Added custom layer editing. Fixes #156

WIP #156 Updated rootviz component config

WIP #156 Added template creation from parent if no code

WIP #156 Added root viz panel. Changed ordering

WIP #156 Added some support for 'create layer' button

WIP #156 Fixed create custom layer type button

WIP #156 Fixed bug with checking node for attribute

WIP #156. Added support for updating meta from constructor args

WIP #156

WIP #156 Added seeds

WIP #156 Added attr inheritance in layer def

WIP #156 Removed extra TODO

* Added custom layer support to executions. Fixes #296

WIP #296 Fixed execution support

* Updated GenArch for custom layers. Fixes #157

WIP #157 Updated 'standalone' default to true

WIP #157 Fixed code-climate issues

WIP #157 Fixed bad default

WIP Removed outdated tests
2016-06-17 13:28:50 -05:00
Brian Broll b65c0b740a Revert "Added regex to remove console colors. Fixes #306 " (#311) 2016-06-16 15:07:18 -05:00
JimnyCricket 55d8b72e05 Added regex to remove console colors. Fixes #306 (#307) 2016-06-16 14:18:34 -05:00
JimnyCricket 1d4469d2b9 Merge pull request #309 from dfst/308-update-nn-seed
Updated project, nn, and minimal seeds. Fixes #308
2016-06-16 14:00:12 -05:00
Brian Broll fda5be793d Updated project, nn, and minimal seeds. Fixes #308 2016-06-16 11:15:49 -05:00
Brian Broll a8666959c0 Added download execFiles button. Fixes #295
WIP #295. updated seeds
2016-06-16 07:58:54 -05:00
Brian Broll f95125ab04 Removed ruler from LogViewer. Fixes #302 (#303) 2016-06-16 06:12:13 -05:00
Brian Broll a5cbfb2c89 Setting stdout from results on job finished. Fixes #300 (#301) 2016-06-16 06:06:23 -05:00
Brian Broll b43a580683 Added node updating to CreateTorchMeta. Fixes #290
WIP #290 Added attribute updating and meta sheet placement

WIP #290 Removing old layers and changing category support

WIP #290 Added update layer test

WIP #290 Updated ArchEditor layer type colors

WIP #290 Updated seeds

WIP #290 Fixed attribute not showing up bug

WIP #290 Fixed removal of old attribute names

WIP #290 Added more logs

WIP #290 Removed 'only' from tests
2016-06-15 21:10:27 -05:00
Brian Broll 1f0f615fc4 Added stdout viewer for jobs. Fixes #162 (#294)
WIP #162. Hid 'success' and 'debug' attrs

WIP #162 Added stdout logs to jobs

WIP #162 Added 'Enter' button on jobs

WIP #162 Added readonly, no syntax highlighting to LogViewer

WIP #162 Added theme

WIP #162. Removed extra new lines

WIP #162. redirected stderr to stdout

WIP #162 Updated seeds

WIP #162 Removed unnecessary comment
2016-06-15 20:39:27 -05:00
Brian Broll 3842439112 Added port update on node creation, port change. Fixes #258 (#282)
WIP #258 Works with artifact loader updates

WIP #258. Fixed reconnection for port update
2016-06-15 13:31:33 -05:00
Brian Broll 8229495b34 Changed downcasting to set base and copy content. Fixes #283 (#284) 2016-06-15 13:31:19 -05:00
Brian Broll b0135100ed Added parsing torch nn for nn visual lang. Fixes #85 (#291)
WIP 85 Added Parser

WIP #85 Fixed BatchNormalization bug and some styling

WIP #85 Added types and json conversion.

WIP #85 create nn layers from parsed library.

WIP #85 Skipping abstract torch layer types. Updated categories

WIP #85 Filtered out 'Criterion' type. Update category name for pos

WIP #85 Added 'baseType' to layer description

WIP #85 Added init params inheritance

WIP #85 Removed out-dated tests

WIP #85 Updated GenArch to omit unset values
2016-06-15 11:27:01 -05:00
Brian Broll a628256682 Set ArtifactLoader name to the target name. Fixes #288 (#289) 2016-06-14 14:03:52 -05:00
Brian Broll ddd6431d55 Added port name tooltips. Fixes #286 (#287) 2016-06-14 13:09:40 -05:00
Brian Broll a63287df92 Removed git lfs from readme; added links 2016-06-14 10:47:09 -05:00
Brian Broll 3721257559 Removed git lfs Fixes #153 (#285)
* WIP #153 Removed git lfs

* WIP #153 Added project seeds back to git
2016-06-14 10:42:57 -05:00
Brian Broll dba9c4a25b Added meta positioning for CreateTorchMeta. Fixes #41. Fixes #24 (#281) 2016-06-14 07:45:49 -05:00
Brian Broll 11f7751843 Destory editor on viz close. Fixes #184 (#280) 2016-06-13 18:30:05 -05:00
Brian Broll c05d7bb433 Added ArtifactFinder. Fixes #236 (#277)
WIP #236 Added ArtifactFinder support to ExecutePipeline

WIP #236 Updated UI for ArtifactFinder support

WIP #236 updated seeds
2016-06-13 13:09:10 -05:00
Brian Broll 8e6bdfbeff Added time limit for upload toast. Fixes #274 (#275) 2016-06-13 11:52:57 -05:00
Brian Broll b4829adbc3 v0.5.0
Updated state badge
2016-06-13 10:58:33 -05:00
Brian Broll a3cfa9f8e3 Added concat support to GenArch. Fixes #61 (#273)
WIP #61 Started w/ concat support

WIP #61 Added basic concat support

WIP #61 Added quick merge-fork fn-ality

WIP #64 Added nested concat support

WIP #61 Added concat tests

WIP #61 Added `npm run watch-test` cmd

WIP #61 Fixed lint errors. Removed debugger statement
2016-06-13 10:53:52 -05:00
Brian Broll 62a80d1f9f Set the logo font for just the item-label. Fixes #271 (#272) 2016-06-11 23:09:53 -05:00
Brian Broll b0ad46c66b Hide fab on no actions. Fixes #269 (#270) 2016-06-11 22:30:51 -05:00
Brian Broll 7e2b83b5cc Added font for deepforge logo. Fixes #267 (#268) 2016-06-11 18:29:47 -05:00
JimnyCricket 5193bbd931 Merge pull request #266 from dfst/259-deleting-input-to-operation
Added removePort to OperationNode. Fixes #259
2016-06-11 18:19:37 -05:00
Brian Broll b513bcbec4 Added removePort to OperationNode. Fixes #259 2016-06-11 18:08:32 -05:00
JimnyCricket 408ee8e0f4 Updated dist code gen to support nil inputs, refs Fixes #228 (#239)
WIP #228 Add nil return to deserialize fn

WIP #228 Removed empty object hashes

WIP #228 Added check for no filename

WIP #228 Updated dist code gen for pointers, inputs

WIP #228. Removed unnecessary template stuff and added warning

WIP #228. local nil var shadowing if no input/ptr
2016-06-11 13:53:49 -05:00
Brian Broll dd92726044 Added torch dep check on worker start. Fixes #255 (#265) 2016-06-11 12:45:48 -05:00
Brian Broll 5d2098e6e1 Removed webgme logo and text. Fixes #260 (#264) 2016-06-11 12:12:25 -05:00
Brian Broll 341497323c Removed default webgme footer. Fixes #262 (#263) 2016-06-11 12:12:22 -05:00
Brian Broll 1f71b254b5 Added require 'nn' to model deser. Fixes #252 (#253) 2016-06-10 13:15:46 -05:00
Brian Broll db97a492e1 Added sorted, colored, root cards w/ icons. Fixes #238 (#251)
WIP #238 Added root viz icons... Needs ranking

WIP #238 Fixed ordering

WIP #238 Added colors

WIP #238 Extended desc of pipelines to fix formatting
2016-06-10 09:48:58 -05:00
Brian Broll f863fc2638 Merge branch '249-multiple-outputs-exec' 2016-06-10 09:00:43 -05:00
JimnyCricket 3d6c654028 Merge pull request #248 from dfst/237-execution-forks
Provided a 'fork-free' save for pipeline execution. Fixes #237
2016-06-10 08:59:14 -05:00
Brian Broll 64f64ecd2a Fixed operation input double counting. Fixes #249 2016-06-10 08:57:48 -05:00
Brian Broll 09b33792d9 Provided a 'fork-free' save for pipeline execution. Fixes #237 2016-06-10 08:24:01 -05:00
Brian Broll ab932c1a52 Fixed case in dependency. Fixes #241 (#242) 2016-06-09 20:49:47 -05:00
Brian Broll 206f4ce6fc Added git-lfs to install instructions. Fixes #240
Also updated some style rules in eslint
2016-06-09 08:28:43 -05:00
JimnyCricket 1618460984 Merge pull request #234 from dfst/232-jump-to-op-def
Added pivoting btwn op defs and pipeline. Fixes #232
2016-06-08 22:34:58 -05:00
Brian Broll 41cfdfb007 Added ArtifactLoader. Fixes #166 (#235)
WIP #166 Added ArtifactLoader to seeds

WIP #166 Added OpIntDecorator

WIP #166 Filtered out the meta data nodes

WIP #166 Added node creation/base change on pointer set

WIP #166. Fixed output node creation

WIP #166 Added ArtifactLoader to LocalExecutor
2016-06-08 11:10:16 -05:00
Brian Broll 57a60d6919 Added pivoting btwn op defs and pipeline. Fixes #232
WIP #232 Added jump to op def button in selection

WIP #232 Added returning to pipeline
2016-06-08 10:29:05 -05:00
Brian Broll ab3ba50886 Added 'upload artifact' button. Fixes #225 (#227) 2016-06-07 14:32:48 -05:00
JimnyCricket cc21727a7b Update README.md
add npm
2016-06-07 12:43:46 -05:00
JimnyCricket 1dbbf8a302 Update README.md
Combining lines.
2016-06-07 12:41:24 -05:00
JimnyCricket 6f6ab1af1e Update README.md
collapse lines
2016-06-07 12:32:37 -05:00
JimnyCricket 09a51dd518 Update README.md
Don't start mongo in sudo.
2016-06-07 12:31:55 -05:00
JimnyCricket 3e0b1c0e50 Update README.md
Added Quick Setup
2016-06-07 12:26:17 -05:00
JimnyCricket 628c636129 Update README.md
Moved Torch installation to Installation Guide
2016-06-07 12:18:38 -05:00
JimnyCricket d2173bd9f3 Update README.md
Referenced the Installation Guide
2016-06-07 12:17:44 -05:00
JimnyCricket 2baf4a72bf Merge pull request #213 from dfst/207-del-op-ptr
207 del op ptr
2016-06-07 11:16:30 -05:00
Brian Broll de1ca7a2d2 Renamed ROOT -> HOME. Fixes #109 (#226) 2016-06-07 10:44:45 -05:00
Brian Broll 2b75f16b9a Added referencing architectures. Fixes #222 (#223) 2016-06-07 10:18:23 -05:00
Brian Broll 20618ddb40 Added intermittent commits to pipeline execution. Fixes #150 (#220)
WIP #150. Added commits...

WIP #150 cancelled commits...?

WIP #150 Creating jobs serially. Fixed cancelled commits

WIP #150 Removed unneeded comment
2016-06-06 11:22:53 -05:00
JimnyCricket be97f4806e Merge pull request #217 from dfst/205-slow-polling
Shortened worker refresh in 'local' mode. Fixes #205
2016-06-06 10:56:18 -05:00
JimnyCricket cb97a14e59 Merge pull request #210 from dfst/154-basic-torch-data-types
Added data types to project and added minimal seed. Yeah, I think having some connections to the blob might be good. We'll address these in another issue. Fixes #154
2016-06-06 10:49:18 -05:00
JimnyCricket 6faa7d444a Merge pull request #218 from dfst/214-code-editor-indents
set tab size to 3. Fixes #214
2016-06-06 10:38:29 -05:00
Brian Broll 806780bd1f Added single execution file generation. Fixes #138 (#219)
WIP #138 Added basic code generation for each op.

WIP #138 merged operation code blocks in topo ordering

WIP #138 Removed unused file and updated plugin name

WIP #138 Added support for references

WIP #138. Updated plugin for executions

WIP #138 Fixed ptr code generation

WIP #138 Updated pipeline seed
2016-06-06 09:56:59 -05:00
Brian Broll 2e0ddfaeb9 set tab size to 3. Fixes #214 2016-06-04 13:29:09 -05:00
Brian Broll c83d4ecbc5 Shortened worker refresh in 'local' mode. Fixes #205 2016-06-04 11:15:21 -05:00
Brian Broll a467e67524 Merge branch 'master' into 207-del-op-ptr 2016-06-04 09:45:10 -05:00
Brian Broll 271f237eac Merge branch 'master' into 154-basic-torch-data-types 2016-06-04 09:44:53 -05:00
Brian Broll 857be35efa Updated Delete button for ptrs. Fixes #207 2016-06-04 09:38:57 -05:00
Brian Broll 87f73359ab Added data types to project and added minimal seed. Fixes #154 2016-06-04 08:59:39 -05:00
151 arquivos alterados com 15619 adições e 1208 exclusões
+35
Ver Arquivo
@@ -0,0 +1,35 @@
---
engines:
csslint:
enabled: true
duplication:
enabled: true
exclude_fingerprints:
- 1e004cf4e49528a58a0ac3858112601c
config:
languages:
- ruby
- javascript
- python
- php
eslint:
enabled: true
fixme:
enabled: true
ratings:
paths:
- "**.css"
- "**.inc"
- "**.js"
- "**.jsx"
- "**.module"
- "**.php"
- "**.py"
- "**.rb"
exclude_paths:
- config/
- test/
- src/common/lua.js
- src/common/js-yaml.min.js
- src/visualizers/widgets/TextEditor/lib/
- src/visualizers/widgets/PipelineIndex/styles/PipelineIndex.css
+3
Ver Arquivo
@@ -0,0 +1,3 @@
--exclude-exts=.min.css
--exclude-list=src/visualizers/widgets/PipelineIndex/styles/PipelineIndex.css
--ignore=adjoining-classes,box-model,ids,order-alphabetical,unqualified-attributes
+1
Ver Arquivo
@@ -2,3 +2,4 @@ src/common/lua.js
src/visualizers/widgets/TextEditor/lib/*
src/common/js-yaml.min.js
src/visualizers/Visualizers.json
config/config.webgme.js
+5 -4
Ver Arquivo
@@ -1,18 +1,19 @@
env:
browser: true
node: true
mocha: true
es6: true
extends: 'eslint:recommended'
rules:
indent:
- error
- 2
- 4
linebreak-style:
- error
- 2
- unix
quotes:
- error
- 2
- single
semi:
- error
- 2
- always
+1 -1
Ver Arquivo
@@ -1 +1 @@
*.webgmex filter=lfs diff=lfs merge=lfs -text
+12 -7
Ver Arquivo
@@ -1,22 +1,27 @@
[![Release State](https://img.shields.io/badge/state-pre--alpha-red.svg)](https://img.shields.io/badge/state-pre--alpha-red.svg)
[![Release State](https://img.shields.io/badge/state-beta-yellow.svg)](https://img.shields.io/badge/state-beta-yellow.svg)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](./LICENSE)
[![Build Status](https://travis-ci.org/dfst/deepforge.svg?branch=master)](https://travis-ci.org/dfst/deepforge)
[![Join the chat at https://gitter.im/dfst/deepforge](https://badges.gitter.im/dfst/deepforge.svg)](https://gitter.im/dfst/deepforge?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
[![Stories in Ready](https://badge.waffle.io/dfst/deepforge.png?label=ready&title=Ready)](https://waffle.io/dfst/deepforge)
**Notice**: DeepForge is still a work in progress and is also lacking significant documentation! That being said, any contributions and/or feedback is greatly appreciated (and feel free to always ask any questions on the gitter)!
# DeepForge
DeepForge is an open-source visual development environment for deep learning. Currently, it supports Convolutional Neural Networks but we are planning on supporting additional deep learning classifiers such as RNNs and LSTMs. Additional features include real-time collaborative editing and version control.
## Quick Setup
After cloning the repo, run
## Quick Start
Simply run the following command to install deepforge with its dependencies:
```
npm install && npm start
curl -o- https://raw.githubusercontent.com/dfst/deepforge/master/install.sh | bash
```
Now, navigate to `localhost:8888` in a browser and create a new project. Select `nn` as the seed and start creating your neural nets!
Next, follow the postinstall instructions to start MongoDB and DeepForge!
Finally, navigate to [http://localhost:8888](http://localhost:8888) to start using DeepForge! For more, detailed instructions,check out our [wiki](https://github.com/dfst/deepforge/wiki/Installation-Guide).
## Caffe Support?
DeepForge uses Torch to perform the actual training and testing of the models. If you are interested in DeepForge using Caffe for actual training and testing, check out [DeepForge-Caffe](https://github.com/dfst/deepforge-caffe).
## Interested in contributing?
Contributions are welcome! Either fork the project and submit some PR's or shoot me an email about getting more involved!
Sponsored by [Digital Reasoning](http://www.digitalreasoning.com/)
+1
Ver Arquivo
@@ -4,6 +4,7 @@ var spawn = require('child_process').spawn,
execJob,
workerJob = null;
process.env.NODE_ENV = 'local';
execJob = spawn('npm', [
'start'
]);
+13 -3
Ver Arquivo
@@ -1,8 +1,9 @@
'use strict';
/*globals process, __dirname, require*/
var path = require('path'),
fs = require('fs'),
spawn = require('child_process').spawn,
childProcess = require('child_process'),
spawn = childProcess.spawn,
projectConfig = require(__dirname + '/../config'),
executorSrc = path.join(__dirname, '..', 'node_modules', 'webgme', 'src',
'server', 'middleware', 'executor', 'worker'),
@@ -12,13 +13,22 @@ var path = require('path'),
address,
config = {};
// Check torch support
var result = childProcess.spawnSync('th', ['--help']);
if (result.error) {
console.error('Checking Torch7 dependency failed. Do you have Torch7 installed ' +
'and in your PATH?\n\nFor Torch7 installation instructions, check out ' +
'http://torch.ch/docs/getting-started.html');
process.exit(1);
}
var startExecutor = function() {
// Start the executor
var execJob = spawn('node', [
'node_worker.js',
workerConfigPath,
workerTmp
]);
]);
execJob.stdout.pipe(process.stdout);
execJob.stderr.pipe(process.stderr);
};
+31 -14
Ver Arquivo
@@ -1,8 +1,26 @@
{
"AutoViz": {
"preloadIds": [
"ArchEditor",
"PipelineEditor",
"OperationEditor",
"ExecutionView"
]
},
"ArchEditor": {
"hotkeys": "none",
"LayerColors": {}
},
"BreadcrumbHeader": {
"pathRule": "history"
},
"FloatingActionButton": {
"hideOnEmpty": true
},
"GenericUIProjectNavigatorController": {
"rootMenuClass": "deepforge-logo",
"rootDisplayName": "DeepForge"
},
"CHFLayout": {
"panels": [
{
@@ -13,7 +31,7 @@
},
{
"id": "Footer",
"panel": "FooterControls/FooterControlsPanel",
"panel": "Footer/FooterPanel",
"container": "footer",
"DEBUG_ONLY": false
},
@@ -36,27 +54,26 @@
"nodes": [
{
"nodeName": "MyArchitectures",
"title": "Architectures",
"icon": "shuffle",
"rank": 1,
"description": "Neural network architectures are stored here and can be used in pipelines."
},
{
"nodeName": "MyExecutions",
"description": "Executions are read-only snapshots of pipelines that have been executed. Past and current executing pipelines are stored here."
},
{
"nodeName": "MyPipelines",
"description": "Pipelines are used for training, testing and ensembling models."
},
{
"nodeName": "MyOperations",
"description": "Operations are the building blocks of pipelines. Custom operations can be created and stored here."
"title": "Pipelines",
"color": "blue-grey",
"icon": "input",
"rank": 3,
"description": "Pipelines compose operations together to effectively train, test and/or ensemble models."
},
{
"nodeName": "MyArtifacts",
"title": "Artifacts",
"icon": "view_quilt",
"color": "blue-grey",
"rank": 5,
"description": "Artifacts from pipeline executions are stored here."
},
{
"nodeName": "MyDataTypes",
"description": "Custom defined data types are stored here."
}
]
}
+6 -2
Ver Arquivo
@@ -4,9 +4,10 @@
var config = require('./config.webgme'),
validateConfig = require('webgme/config/validator');
require('dotenv').load();
require('dotenv').load({silent: true});
// Add/overwrite any additional settings here
config.server.port = process.env.PORT || config.server.port;
config.server.port = +process.env.PORT || config.server.port;
config.mongo.uri = process.env.MONGO_URI || config.mongo.uri;
config.requirejsPaths.deepforge = './src/common';
config.seedProjects.defaultProject = 'project';
@@ -15,6 +16,9 @@ config.plugin.allowBrowserExecution = true;
config.plugin.allowServerExecution = true;
config.executor.enable = true;
config.executor.clearOldDataAtStartUp = true;
config.visualization.extraCss.push('deepforge/styles/global.css');
validateConfig(config);
module.exports = config;
+1 -2
Ver Arquivo
@@ -9,8 +9,7 @@ var config = require('./config.base'),
// config.mongo.uri = 'mongodb://127.0.0.1:27017/webgme_my_app';
// Seeds for development are prefixed with 'dev'
config.seedProjects.basePaths = config.seedProjects.basePaths
.filter(path => path.indexOf('dev') === -1);
config.seedProjects.basePaths = ['src/seeds/project', 'src/seeds/cifar10'];
validateConfig(config);
module.exports = config;
+13
Ver Arquivo
@@ -0,0 +1,13 @@
// Config for running deepforge w/ one local worker
// jshint node: true
'use strict';
var config = require('./config.default'),
validateConfig = require('webgme/config/validator');
// Turn up the worker polling rate
config.executor.workerRefreshInterval = 150;
config.executor.clearOldDataAtStartUp = true,
validateConfig(config);
module.exports = config;
+1 -2
Ver Arquivo
@@ -15,12 +15,11 @@ config.visualization.decoratorPaths.push('src/decorators');
config.visualization.decoratorPaths.push('node_modules/webgme-easydag/src/decorators');
config.seedProjects.basePaths.push('src/seeds/nn');
config.seedProjects.basePaths.push('src/seeds/devTests');
config.seedProjects.basePaths.push('src/seeds/devMinimal');
config.seedProjects.basePaths.push('src/seeds/devUtilTests');
config.seedProjects.basePaths.push('src/seeds/pipeline');
config.seedProjects.basePaths.push('src/seeds/devPipelineTests');
config.seedProjects.basePaths.push('src/seeds/demo');
config.seedProjects.basePaths.push('src/seeds/project');
config.seedProjects.basePaths.push('src/seeds/cifar10');
+126
Ver Arquivo
@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# Things to install:
# - nvm
command -v git >/dev/null 2>&1 || { echo >&2 "I require git but it's not installed. Aborting."; exit 1; }
echo >&2 "Checking DeepForge dependencies...";
command -v th >/dev/null 2>&1 || {
# No torch!
echo >&2 "Torch is not found. Installing...";
git clone https://github.com/torch/distro.git ~/torch --recursive;
cd ~/torch; bash install-deps;
./install.sh;
}
# profile (bash, zsh, profile, etc) borrowed from nvm's installer
detect_profile() {
if [ -n "$PROFILE" -a -f "$PROFILE" ]; then
echo "$PROFILE"
return
fi
DETECTED_PROFILE=''
local SHELLTYPE
SHELLTYPE="$(basename "/$SHELL")"
if [ "$SHELLTYPE" = "bash" ]; then
if [ -f "$HOME/.bashrc" ]; then
DETECTED_PROFILE="$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
DETECTED_PROFILE="$HOME/.bash_profile"
fi
elif [ "$SHELLTYPE" = "zsh" ]; then
DETECTED_PROFILE="$HOME/.zshrc"
fi
if [ -z "$DETECTED_PROFILE" ]; then
if [ -f "$HOME/.profile" ]; then
DETECTED_PROFILE="$HOME/.profile"
elif [ -f "$HOME/.bashrc" ]; then
DETECTED_PROFILE="$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
DETECTED_PROFILE="$HOME/.bash_profile"
elif [ -f "$HOME/.zshrc" ]; then
DETECTED_PROFILE="$HOME/.zshrc"
fi
fi
}
detect_profile
command -v node >/dev/null 2>&1 || {
# No node! Install nvm
echo >&2 "NodeJS is not found. Installing (using nvm)...";
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.1/install.sh | bash;
source $DETECTED_PROFILE
. $NVM_DIR/nvm.sh
# Install nodejs v6.2.0
echo "Installing nodejs v6.2.0"
nvm install v6.2.0
nvm alias default v6.2.0
# Install npm@2
npm install npm@2 -g
}
command -v node >/dev/null 2>&1 || {
# No mongod!
echo >&2 "MongoDB is not found. Installing...";
if [[ `uname` == "Darwin" ]]; then
brew install mongodb
elif [[ "$(uname)" == 'Linux' ]]; then
if [[ -r /etc/os-release ]]; then
# this will get the required information without dirtying any env state
DIST_VERS="$( ( . /etc/os-release &>/dev/null
echo "$ID $VERSION_ID") )"
DISTRO="${DIST_VERS%% *}" # get our distro name
VERSION="${DIST_VERS##* }" # get our version number
elif [[ -r /etc/lsb-release ]]; then
DIST_VERS="$( ( . /etc/lsb-release &>/dev/null
echo "${DISTRIB_ID,,} $DISTRIB_RELEASE") )"
DISTRO="${DIST_VERS%% *}" # get our distro name
VERSION="${DIST_VERS##* }" # get our version number
else # well, I'm out of ideas for now
echo '==> Failed to determine distro and version.'
exit 1
fi
# Detect archlinux
if [[ "$DISTRO" = "arch" ]]; then
distribution="archlinux"
sudo pacman -S mongodb
# Detect Ubuntu
elif [[ "$DISTRO" = "ubuntu" ]]; then
export DEBIAN_FRONTEND=noninteractive
sudo apt-get install mongodb
else
NEEDS_MONGO=true
fi
fi
}
echo >&2 "Installing DeepForge...";
# Clone deepforge into ~/deepforge
git clone https://github.com/dfst/deepforge ~/deepforge
cd ~/deepforge
npm install
mkdir ~/deepforge/data 2> /dev/null
echo "Final Installation steps:"
echo " 1) Close and re-open your terminal"
echo " (or run \"source $DETECTED_PROFILE\")"
if [[ $NEEDS_MONGO ]]; then
echo " 2) Install MongoDB for your OS"
echo " (available at https://www.mongodb.com/download-center)"
fi
echo ""
echo "Then run DeepForge!"
echo " 1) make sure MongoDB is running locally"
echo " (start mongo w/ \"mongod --dbpath ~/deepforge/data\")"
echo " 2) Run \"npm run local\" from ~/deepforge"
+9 -6
Ver Arquivo
@@ -3,20 +3,23 @@
"scripts": {
"start": "node app.js",
"start-dev": "NODE_ENV=dev node app.js",
"worker": "node ./bin/start-worker.js",
"local": "node ./bin/start-local.js",
"test": "node ./node_modules/mocha/bin/mocha --recursive test"
"worker": "node ./bin/start-worker.js",
"test": "node ./node_modules/mocha/bin/mocha --recursive test",
"watch-test": "./node_modules/nodemon/bin/nodemon.js --exec 'node ./node_modules/mocha/bin/mocha --recursive test'",
"build-nn": "node ./utils/nn-parser.js"
},
"version": "0.4.0",
"version": "0.9.0",
"dependencies": {
"dotenv": "^2.0.0",
"lodash.difference": "^4.1.2",
"nodemon": "^1.9.2",
"webgme": "^2.0.0",
"webgme-autoviz": "^2.0.3",
"webgme-breadcrumbheader": "^2.0.0",
"webgme-autoviz": "dfst/webgme-autoviz",
"webgme-breadcrumbheader": "^2.1.0",
"webgme-chflayout": "^2.0.0",
"webgme-easydag": "dfst/webgme-easydag",
"webgme-fab": "^2.0.2",
"webgme-fab": "dfst/webgme-fab",
"webgme-simple-nodes": "^2.0.0"
},
"devDependencies": {
+294
Ver Arquivo
@@ -0,0 +1,294 @@
/* globals Materialize, WebGMEGlobal, define*/
// This file creates the DeepForge namespace and defines basic actions
define([
'js/RegistryKeys',
'js/Panels/MetaEditor/MetaEditorConstants',
'js/Constants'
], function(
REGISTRY_KEYS,
META_CONSTANTS,
CONSTANTS
) {
var DeepForge = {},
placesTerritoryId,
client = WebGMEGlobal.Client,
PLACE_NAMES;
// Helper functions
var addToMetaSheet = function(nodeId, metasheetName) {
var root = client.getNode(CONSTANTS.PROJECT_ROOT_ID),
metatabs = root.getRegistry(REGISTRY_KEYS.META_SHEETS),
metatab = metatabs.find(tab => tab.title === metasheetName) || metatabs[0],
metatabId = metatab.SetID;
// Add to the general meta
client.addMember(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
META_CONSTANTS.META_ASPECT_SET_NAME
);
client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
META_CONSTANTS.META_ASPECT_SET_NAME,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
// Add to the specific sheet
client.addMember(CONSTANTS.PROJECT_ROOT_ID, nodeId, metatabId);
client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
metatabId,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
};
var createNamedNode = function(baseId, parentId, isMeta) {
var newId = client.createChild({parentId, baseId}),
baseNode = client.getNode(baseId),
basename = 'New' + baseNode.getAttribute('name'),
newName = getUniqueName(parentId, basename);
// If instance, make the first char lowercase
if (!isMeta) {
newName = newName.substring(0, 1).toLowerCase() + newName.substring(1);
}
// Set isAbstract false, if needed
if (baseNode.getRegistry('isAbstract')) {
client.setRegistry(newId, 'isAbstract', false);
}
client.setAttributes(newId, 'name', newName);
return newId;
};
var getUniqueName = function(parentId, basename) {
var pNode = client.getNode(parentId),
children = pNode.getChildrenIds().map(id => client.getNode(id)),
name = basename,
exists = {},
i = 2;
children.forEach(child => exists[child.getAttribute('name')] = true);
while (exists[name]) {
name = basename + '_' + i;
i++;
}
return name;
};
//////////////////// DeepForge places detection ////////////////////
var TYPE_TO_CONTAINER = {
Architecture: 'MyArchitectures',
Pipeline: 'MyPipelines',
Execution: 'MyExecutions',
Layer: 'MyLayers',
Artifact: 'MyArtifacts',
Operation: 'MyOperations',
Primitive: 'MyDataTypes',
Complex: 'MyDataTypes'
};
PLACE_NAMES = Object.keys(TYPE_TO_CONTAINER).map(key => TYPE_TO_CONTAINER[key]);
// Add DeepForge directories
var updateDeepForgeNamespace = function() {
var territory = {};
DeepForge.places = {};
// Create a territory
if (placesTerritoryId) {
client.removeUI(placesTerritoryId);
}
territory[CONSTANTS.PROJECT_ROOT_ID] = {children: 1};
placesTerritoryId = client.addUI(null, updateDeepForgePlaces);
// Update the territory (load the main places)
client.updateTerritory(placesTerritoryId, territory);
};
var updateDeepForgePlaces = function(events) {
var nodeIdsByName = {},
nodes;
nodes = events
// Remove root node, complete event and update/unload events
.filter(event => event.eid && event.eid !== CONSTANTS.PROJECT_ROOT_ID)
.filter(event => event.etype === CONSTANTS.TERRITORY_EVENT_LOAD)
.map(event => client.getNode(event.eid));
nodes.forEach(node =>
nodeIdsByName[node.getAttribute('name')] = node.getId());
PLACE_NAMES.forEach(name => DeepForge.places[name] = nodeIdsByName[name]);
// Remove the territory
client.removeUI(placesTerritoryId);
placesTerritoryId = null;
};
//////////////////// DeepForge creation actions ////////////////////
var instances = [
'Architecture',
'Pipeline'
],
metaNodes = [
'Operation',
'Primitive',
'Complex'
];
var createNew = function(type, metasheetName) {
var parentId,
placeName = TYPE_TO_CONTAINER[type],
newId,
baseId,
msg = `Created new ${type + (metasheetName ? ' prototype' : '')}`;
baseId = client.getAllMetaNodes()
.find(node => node.getAttribute('name') === type)
.getId();
// Look up the parent container
parentId = DeepForge.places[placeName];
client.startTransaction(msg);
newId = createNamedNode(baseId, parentId, !!metasheetName);
if (metasheetName) {
addToMetaSheet(newId, metasheetName);
}
client.completeTransaction();
WebGMEGlobal.State.registerActiveObject(newId);
return newId;
};
var createCustomLayer = function(typeName) {
var metanodes = client.getAllMetaNodes(),
msg = `Created new custom ${typeName} layer`,
newId,
customLayerId,
baseId,
name,
i = metanodes.length;
while (i-- && !(baseId && customLayerId)) {
name = metanodes[i].getAttribute('name');
if (name === 'CustomLayer') {
customLayerId = metanodes[i].getId();
} else if (name === typeName) {
baseId = metanodes[i].getId();
}
}
client.startTransaction(msg);
newId = createNamedNode(baseId, DeepForge.places.MyLayers, true);
addToMetaSheet(newId, 'CustomLayers');
client.addMixin(newId, customLayerId);
client.setRegistry(newId, REGISTRY_KEYS.IS_ABSTRACT, false);
client.completeTransaction();
WebGMEGlobal.State.registerActiveObject(newId);
};
// Creating Artifacts
var UPLOAD_PLUGIN = 'ImportArtifact',
DATA_TYPE_CONFIG = {
name: 'dataTypeId',
displayName: 'Data Type Id',
valueType: 'string',
valueItems: []
};
var uploadArtifact = function() {
// Get the data types
var dataBase,
dataBaseId,
metanodes = client.getAllMetaNodes(),
dataTypes = [];
dataBase = metanodes.find(n => n.getAttribute('name') === 'Data');
if (!dataBase) {
this.logger.error('Could not find the base Data node!');
return;
}
dataBaseId = dataBase.getId();
dataTypes = metanodes.filter(n => client.isTypeOf(n.getId(), dataBaseId))
.filter(n => !n.getRegistry('isAbstract'))
.map(node => node.getAttribute('name'));
//this.logger.info(`Found ${dataTypes.length} data types`);
// Add the target type to the pluginMetadata... hacky :/
var metadata = WebGMEGlobal.allPluginsMetadata[UPLOAD_PLUGIN],
config = metadata.configStructure
.find(opt => opt.name === DATA_TYPE_CONFIG.name);
if (!config) {
config = DATA_TYPE_CONFIG;
WebGMEGlobal.allPluginsMetadata[UPLOAD_PLUGIN].configStructure.push(config);
}
config.valueItems = dataTypes;
config.value = dataTypes[0];
WebGMEGlobal.InterpreterManager.configureAndRun(metadata, (result) => {
var msg = 'Artifact upload complete!';
if (!result) {
return;
}
if (!result.success) {
msg = `Artifact upload failed: ${result.error}`;
}
Materialize.toast(msg, 2000);
});
};
DeepForge.last = {};
DeepForge.create = {};
instances.forEach(type => {
DeepForge.create[type] = function() {
return createNew.call(null, type);
};
});
metaNodes.forEach(type => {
DeepForge.create[type] = function() {
return createNew.call(null, type, type);
};
});
DeepForge.create.Layer = createCustomLayer;
DeepForge.create.Artifact = uploadArtifact;
//////////////////// DeepForge prev locations ////////////////////
// Update DeepForge on project changed
WebGMEGlobal.State.on('change:' + CONSTANTS.STATE_ACTIVE_PROJECT_NAME,
updateDeepForgeNamespace, null);
// define DeepForge globally
window.DeepForge = DeepForge;
return DeepForge;
});
Diferenças do arquivo suprimidas por serem muito extensas Carregar Diff
+1
Ver Arquivo
@@ -2687,6 +2687,7 @@ function LuaContext(){
}
exports.stdlib(_G, helpers)();
}
this.__helpers = helpers;
}
LuaContext.prototype = {}
@@ -9,9 +9,24 @@ define([
};
// Should these be in lua?
LocalExecutor.prototype.BlobLoader = function(node) {
var hash = this.core.getAttribute(node, 'data');
return this.getOutputs(node)
LocalExecutor.prototype.ArtifactLoader = function(node) {
// Get the hash from the output node
var hash;
return this.core.loadChildren(node)
.then(cntrs => {
// Get the output container and load it's children
var output = cntrs
.find(cntr => {
var metaNode = this.core.getMetaType(cntr),
metaName = this.core.getAttribute(metaNode, 'name');
return metaName === 'Outputs';
});
return this.core.loadChildren(output);
})
.then(dataNodes => {
hash = this.core.getAttribute(dataNodes[0], 'data');
return this.getOutputs(node);
})
.then(outputTuples => {
var outputs = outputTuples.map(tuple => tuple[2]),
paths;
@@ -21,12 +36,52 @@ define([
this.logger.info(`Loading blob data (${hash}) to ${paths.map(p => `"${p}"`)}`);
outputs.forEach(output => this.core.setAttribute(output, 'data', hash));
// Set the metadata as appropriate
// TODO
this.onOperationComplete(node);
});
};
LocalExecutor.prototype.ArtifactFinder = function(node) {
// Check the save dir for a node with the given name
// that has the given type
var hash,
typeId = this.core.getPointerPath(node, 'type'),
type,
artifactName = this.core.getAttribute(node, 'artifactName');
return this.core.loadByPath(this.rootNode, typeId)
.then(_type => {
type = _type;
return this._getSaveDir();
})
.then(saveDir => this.core.loadChildren(saveDir))
.then(artifacts => {
return artifacts.find(artifact =>
this.core.getAttribute(artifact, 'name') === artifactName &&
this.isMetaTypeOf(artifact, type));
})
.then(matchingArtifact => {
hash = matchingArtifact && this.core.getAttribute(matchingArtifact, 'data');
// If no hash, just continue (the subsequent ops will receive 'nil')
if (!hash) {
return this.onOperationComplete(node);
} else {
return this.getOutputs(node)
.then(outputPairs => {
var outputs = outputPairs.map(pair => pair[2]),
paths;
paths = outputs.map(output => this.core.getPath(output));
// Get the 'data' hash and store it in the output data ports
this.logger.info(`Loading blob data (${hash}) to ${paths.map(p => `"${p}"`)}`);
outputs.forEach(output => this.core.setAttribute(output, 'data', hash));
this.onOperationComplete(node);
});
}
});
};
LocalExecutor.prototype._getSaveDir = function () {
return this.core.loadChildren(this.rootNode)
.then(children => {
@@ -90,13 +145,31 @@ define([
this.logger.info(`saving hashes: ${hashes.map(h => `"${h}"`)}`);
this.onOperationComplete(node);
});
// Overwrite existing node w/ this name?
// TODO
};
LocalExecutor.TYPES = Object.keys(LocalExecutor.prototype)
.filter(name => name.indexOf('_') !== 0);
// Helper methods
LocalExecutor.prototype.getLocalOperationType = function(node) {
var type;
for (var i = LocalExecutor.OPERATIONS.length; i--;) {
type = LocalExecutor.OPERATIONS[i];
if (!this.META[type]) {
this.logger.warn(`Missing local operation: ${type}`);
continue;
}
if (this.isMetaTypeOf(node, this.META[type])) {
return type;
}
}
return null;
};
LocalExecutor.prototype.isLocalOperation = function(node) {
return !!this.getLocalOperationType(node);
};
LocalExecutor.OPERATIONS = Object.keys(LocalExecutor.prototype)
.filter(name => name.indexOf('_') !== 0)
.filter(name => name !== 'isLocalOperation' && name !== 'getLocalOperationType');
return LocalExecutor;
});
+48
Ver Arquivo
@@ -0,0 +1,48 @@
/*globals define, WebGMEGlobal*/
define([
'q'
], function(
Q
) {
var PtrCodeGen = function() {
};
PtrCodeGen.prototype.getPtrCodeHash = function(ptrId) {
return this.core.loadByPath(this.rootNode, ptrId)
.then(ptrNode => {
// Look up the plugin to use
var metanode = this.core.getMetaType(ptrNode),
pluginId;
pluginId = this.core.getRegistry(ptrNode, 'validPlugins').split(' ').shift();
this.logger.info(`generating code for ${this.core.getAttribute(ptrNode, 'name')} using ${pluginId}`);
var context = WebGMEGlobal.Client.getCurrentPluginContext(pluginId);
context.managerConfig.namespace = this.core.getNamespace(metanode);
context.managerConfig.activeNode = this.core.getPath(ptrNode);
// Load and run the plugin
return Q.nfcall(this.executePlugin.bind(this), pluginId, context);
})
.then(hashes => hashes[0]); // Grab the first asset for now
};
PtrCodeGen.prototype.executePlugin = function(pluginId, config, callback) {
// Call the Interpreter manager in a Q.ninvoke friendly way
// I need to create a custom context for the given plugin:
// - Set the activeNode to the given referenced node
// - If the activeNode is namespaced, set META to the given namespace
//
// FIXME: Check if it is running in the browser or on the server
WebGMEGlobal.Client.runBrowserPlugin(pluginId, config, (err, result) => {
if (!result.success) {
return callback(result.getError());
}
this.logger.info('Finished calling ' + pluginId);
callback(null, result.artifacts);
});
};
return PtrCodeGen;
});
+24
Ver Arquivo
@@ -0,0 +1,24 @@
/* latin-ext */
@font-face {
font-family: 'Audiowide';
font-style: normal;
font-weight: 400;
src: local('Audiowide'), local('Audiowide-Regular'), url(https://fonts.gstatic.com/s/audiowide/v4/7pSgz2MbVvTCvvm7vukSHxJtnKITppOI_IvcXXDNrsc.woff2) format('woff2');
unicode-range: U+0100-024F, U+1E00-1EFF, U+20A0-20AB, U+20AD-20CF, U+2C60-2C7F, U+A720-A7FF;
}
/* latin */
@font-face {
font-family: 'Audiowide';
font-style: normal;
font-weight: 400;
src: local('Audiowide'), local('Audiowide-Regular'), url(https://fonts.gstatic.com/s/audiowide/v4/8XtYtNKEyyZh481XVWfVOltXRa8TVwTICgirnJhmVJw.woff2) format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
}
.deepforge-logo .item-label {
font-family: 'Audiowide', cursive;
}
.create-node text {
font-style: italic;
}
+65
Ver Arquivo
@@ -0,0 +1,65 @@
/*globals define, WebGMEGlobal*/
define([
'widgets/EasyDAG/Buttons',
'widgets/EasyDAG/Icons'
], function(
EasyDAGButtons,
Icons
) {
// Create a GoToBase button
var client = WebGMEGlobal.Client;
var GoToBase = function(params) {
// Check if it should be disabled
var baseId = this._getBaseId(params.item),
base = baseId && client.getNode(baseId);
if (!params.disabled) {
params.disabled = base ? base.isLibraryElement() : true;
}
EasyDAGButtons.ButtonBase.call(this, params);
};
GoToBase.SIZE = 10;
GoToBase.BORDER = 1;
GoToBase.prototype.BTN_CLASS = 'go-to-base';
GoToBase.prototype = new EasyDAGButtons.ButtonBase();
GoToBase.prototype._render = function() {
var lineRadius = GoToBase.SIZE - GoToBase.BORDER,
btnColor = '#90caf9';
if (this.disabled) {
btnColor = '#e0e0e0';
}
this.$el
.append('circle')
.attr('r', GoToBase.SIZE)
.attr('fill', btnColor);
// Show the 'code' icon
Icons.addIcon('code', this.$el, {
radius: lineRadius
});
};
GoToBase.prototype._onClick = function(item) {
var node = client.getNode(item.id),
baseId = node.getBaseId();
WebGMEGlobal.State.registerActiveObject(baseId);
};
GoToBase.prototype._getBaseId = function(item) {
var n = client.getNode(item.id);
return n && n.getBaseId();
};
return {
DeleteOne: EasyDAGButtons.DeleteOne,
GoToBase: GoToBase
};
});
+3
Ver Arquivo
@@ -0,0 +1,3 @@
.node-prompter .scrollbar {
fill: #bfbfbf;
}
+318
Ver Arquivo
@@ -0,0 +1,318 @@
/*globals define, d3 */
// Given a container and a set of nodes, this will prompt the user
// to select one of the set of nodes. This will also need to support
// adding a "plus" button for creating new objects in line
define([
'q',
'css!./NodePrompter.css'
], function(
Q
) {
var MARGIN = 15,
CLOSING_GRACE = 400,
TRANSITION_DURATION = 400;
var NodePrompter = function(rect, opts) {
opts = opts || {};
// default options
opts.padding = opts.padding || 0;
this.left = rect.left-opts.padding;
this.top = rect.top-opts.padding;
this.width = rect.width + 2*opts.padding;
this.actualHeight = rect.height + 2*opts.padding;
this.height = this.actualHeight; // scroll height
this.cx = opts.cx || rect.left + rect.width/2;
this.cy = opts.cy || rect.top + rect.height/2;
this.active = true;
this.onNode = false;
this.scrollbar = null;
this.scrollPosition = 0;
var container = document.createElement('div');
container.setAttribute('class', 'node-prompter');
this.container = container;
container.style.width = this.width + 'px';
container.style.height = this.height+'px';
container.style.position = 'absolute';
};
NodePrompter.prototype.prompt = function(nodes, selectFn) {
var deferred = Q.defer(),
size,
cornerRadius = 10;
this.selectHandler = selectFn;
this.svg = d3.select(this.container).append('svg')
.attr('width', this.width)
.attr('height', this.height)
.attr('overflow', 'hidden');
document.body.appendChild(this.container);
// Expand the panel
this.panel = this.svg.append('rect');
this.nodeContainer = this.svg.append('g');
size = this.initNodes(nodes);
this.resize(size.width, size.height);
// Create the panel
this.panel
.attr('x', this.cx)
.attr('y', this.cy)
.attr('rx', 1)
.attr('ry', 1)
.attr('height', 1)
.attr('width', 1)
.attr('fill', '#f44336');
this.panel.transition()
.delay(50)
.duration(TRANSITION_DURATION)
.attr('x', 0)
.attr('y', 0)
.attr('rx', cornerRadius)
.attr('ry', cornerRadius)
.attr('height', this.actualHeight)
.attr('width', this.width)
.attr('fill', '#e0e0e0')
.each('end', () => {
// Add the given nodes to the panel
this.showNodes(nodes, deferred.resolve);
// Add scrollbar if height is too large
if (this.height > this.actualHeight) {
this.createScrollbar(cornerRadius);
}
});
// Event handling
this.svg.on('mouseout', () => {
this.active = false;
setTimeout(this.destroyIfInactive.bind(this), CLOSING_GRACE);
});
this.svg.on('mouseover', () => this.active = true);
// Return a promise called on 'selected'
return deferred.promise;
};
NodePrompter.prototype.resize = function(width, height) {
var dx = this.width - width,
maxHeight = this.height,
dy;
this.actualHeight = Math.min(maxHeight, height);
dy = this.height - this.actualHeight;
this.nodes.forEach(node => node.moveBy(-dx/2, 0));
this.left += dx;
this.top += dy;
this.cx -= dx;
this.cy -= dy;
this.container.style.left = this.left + 'px';
this.container.style.top = this.top + 'px';
this.width = width;
this.height = height;
};
NodePrompter.prototype.createScrollbar = function(yMargin) {
var width = 4,
actualHeight = this.actualHeight-2*yMargin,
updateScroll = this.updateScroll.bind(this);
// Create the scrollbar
this.scrollBarHeight = actualHeight/this.height*actualHeight;
this.scrollHeight = actualHeight;
this.scrollbar = this.svg.append('rect')
.attr('class', 'scrollbar')
.attr('x', this.width - width)
.attr('y', yMargin)
.attr('rx', 2)
.attr('ry', 2)
.attr('height', this.scrollBarHeight)
.attr('width', width);
// Attach scroll handler to the 'panel' rect
this.svg
.on('zoom', updateScroll)
.on('wheel.zoom', updateScroll)
.on('mousewheel.zoom', updateScroll)
.on('DOMMouseScroll.zoom', updateScroll);
};
NodePrompter.prototype.updateScroll = function() {
var delta = d3.event.deltaY,
sensitivity = 1,
maxScroll = this.scrollHeight - this.scrollBarHeight,
containerY,
relView;
this.scrollPosition += delta*sensitivity;
this.scrollPosition = Math.max(this.scrollPosition, 0);
this.scrollPosition = Math.min(this.scrollPosition, maxScroll);
this.scrollbar
.attr('transform', `translate(0, ${this.scrollPosition})`);
// Update the translation on the nodeContainer
relView = this.scrollPosition/maxScroll;
containerY = relView * (this.height-this.actualHeight);
this.nodeContainer
.attr('transform', `translate(0, -${containerY})`);
};
NodePrompter.prototype.destroyIfInactive = function() {
// Verify that is not over any of the displayed nodes
if (!this.active && !this.onNode) {
this.destroy();
}
};
NodePrompter.prototype.destroy = function() {
this.hideNodes();
if (this.scrollbar) {
this.scrollbar.remove();
}
this.panel.transition()
.duration(TRANSITION_DURATION)
.attr('x', this.cx)
.attr('y', this.cy)
.attr('rx', 1)
.attr('ry', 1)
.attr('height', 1)
.attr('width', 1)
.attr('fill', '#f44336')
.each('end', () => {
this.container.remove();
});
};
NodePrompter.prototype.onSelected = function(container, callback) {
// Return the id
if (this.selectHandler) {
this.selectHandler(container.node, this);
} else {
this.destroy();
return callback(container.node);
}
};
NodePrompter.prototype.initNodes = function(nodes) {
// For each node, create the containers and position them
var decorators = nodes.map(node => new Container(this.nodeContainer, node)),
lineGroup,
maxLineWidth = this.width - 2*MARGIN,
totalWidth = 0,
lineWidth,
lineStartHeight = MARGIN,
lineHeight,
cntr,
x,y,
i = 0;
// Position the nodes. while we can fit the node on the given line, add it
decorators.forEach(d => {
d.computeSize(0.25);
});
while (i < decorators.length) {
lineGroup = [decorators[i]];
lineWidth = decorators[i].width() + MARGIN;
lineHeight = decorators[i].height();
i++;
while (i < decorators.length &&
lineWidth + decorators[i].width() + MARGIN < maxLineWidth) {
lineGroup.push(decorators[i]);
lineWidth += decorators[i].width() + MARGIN;
lineHeight = Math.max(lineHeight, decorators[i].height());
i++;
}
// Get the positions for each
lineWidth += MARGIN;
totalWidth = Math.max(lineWidth, totalWidth);
x = (this.width-lineWidth)/2 + MARGIN;
for (var g = 0; g < lineGroup.length; g++) {
cntr = lineGroup[g];
y = (lineHeight - cntr.height())/2 + lineStartHeight;
cntr.goTo(x, y);
x += cntr.width() + MARGIN;
}
lineStartHeight += lineHeight + MARGIN;
}
this.nodes = decorators;
return {
height: lineStartHeight,
width: totalWidth
};
};
NodePrompter.prototype.showNodes = function(nodes, callback) {
this.nodes.forEach(d => {
d.render(0.25);
d.$el.on('mouseover', () => this.onNode = true);
d.$el.on('mouseout', () => this.onNode = false);
d.$el.on('click', () => this.onSelected(d, callback));
});
};
NodePrompter.prototype.hideNodes = function() {
this.nodes.forEach(node => node.$el.remove());
};
var Container = function(svg, node) { // used for positioning
this.$el = svg.append('g');
this.x = 0;
this.y = 0;
this.node = node;
this.decorator = new node.Decorator({
node: node,
parentEl: this.$el
});
};
Container.prototype.moveBy = function(dx, dy) {
dx = dx || 0;
dy = dy || 0;
this.x += dx;
this.y += dy;
};
Container.prototype.goTo = function(x, y) {
this.x = x;
this.y = y;
};
Container.prototype.computeSize = function(zoom) {
this.$el.attr('opacity', 0);
this.decorator.render(zoom);
};
Container.prototype.render = function(zoom) {
this.$el.attr('transform', `translate(${this.x}, ${this.y})`);
this.$el
.transition()
.attr('opacity', 1);
this.decorator.render(zoom);
};
Container.prototype.width = function() {
return this.decorator.width;
};
Container.prototype.height = function() {
return this.decorator.height;
};
return NodePrompter;
});
+2 -2
Ver Arquivo
@@ -7,9 +7,9 @@ define([
var OperationControl = function() {
};
OperationControl.prototype.hasMetaName = function(id, name) {
OperationControl.prototype.hasMetaName = function(id, name, inclusive) {
var node = this._client.getNode(id),
bId = node.getBaseId(),
bId = inclusive ? id : node.getBaseId(),
baseName;
while (bId) {
+2
Ver Arquivo
@@ -72,6 +72,8 @@ define([
var desc = EasyDAGControl.prototype._getObjectDescriptor.call(this, id),
node = this._client.getNode(id);
desc.inputs = [];
desc.outputs = [];
if (this.hasMetaName(id, 'Operation')) {
// Only decorate operations in the currently active node
if (this._currentNodeId !== desc.parentId) {
+60
Ver Arquivo
@@ -0,0 +1,60 @@
/*globals define*/
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'underscore'
], function(
PanelBaseWithHeader,
IActivePanel,
_
) {
var RenameablePanel = function() {
PanelBaseWithHeader.apply(this, arguments);
};
_.extend(
RenameablePanel.prototype,
PanelBaseWithHeader.prototype,
IActivePanel.prototype
);
RenameablePanel.OPTIONS = PanelBaseWithHeader.OPTIONS;
RenameablePanel.prototype.initializeRenameable = function () {
this.$panelHeaderTitle.on('dblclick', this.editTitle.bind(this));
};
RenameablePanel.prototype.currentNodeId = function () {
return this.control._currentNodeId;
};
RenameablePanel.prototype.currentBaseName = function () {
var currentId = this.currentNodeId(),
node = this._client.getNode(currentId),
baseId = node.getBaseId(),
base = this._client.getNode(baseId);
return base.getAttribute('name');
};
RenameablePanel.prototype.editTitle = function () {
this.$panelHeaderTitle.editInPlace({
css: {
'z-index': 1000
},
onChange: (oldValue, newValue) => {
var nodeId = this.currentNodeId(),
type = this.currentBaseName(),
msg = `Renamed ${type}: ${oldValue} -> ${newValue}`;
if (!/^\s*$/.test(newValue)) {
this._client.startTransaction(msg);
this._client.setAttributes(nodeId, 'name', newValue);
this._client.completeTransaction();
}
}
});
};
return RenameablePanel;
});
@@ -0,0 +1,43 @@
/*globals define, _*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'js/Decorators/DecoratorBase',
'./EasyDAG/ArtifactOpDecorator.EasyDAGWidget'
], function (
DecoratorBase,
ArtifactOpDecoratorEasyDAGWidget
) {
'use strict';
var ArtifactOpDecorator,
__parent__ = DecoratorBase,
__parent_proto__ = DecoratorBase.prototype,
DECORATOR_ID = 'ArtifactOpDecorator';
ArtifactOpDecorator = function (params) {
var opts = _.extend({loggerName: this.DECORATORID}, params);
__parent__.apply(this, [opts]);
this.logger.debug('ArtifactOpDecorator ctor');
};
_.extend(ArtifactOpDecorator.prototype, __parent_proto__);
ArtifactOpDecorator.prototype.DECORATORID = DECORATOR_ID;
/*********************** OVERRIDE DecoratorBase MEMBERS **************************/
ArtifactOpDecorator.prototype.initializeSupportedWidgetMap = function () {
this.supportedWidgetMap = {
EasyDAG: ArtifactOpDecoratorEasyDAGWidget
};
};
return ArtifactOpDecorator;
});
@@ -0,0 +1,43 @@
/**
* @author brollb / https://github.com/brollb
*/
.artifactop-decorator {
min-width: 65px;
height: 40px;
border: 1px solid black;
background-color: #dedede;
padding: 3px;
text-align: center; }
.artifactop-decorator .attr-title {
font-style: italic;
}
.artifactop-decorator .name {
margin-top: 10px;
white-space: nowrap;
font-size: 16px;
font-weight: bold;
font-family: "Helvetica Neue",Helvetica,Arial,sans-serif;
text-align: center; }
.artifactop-decorator .connector {
background-color: #fefefe;
height: 10px;
width: 10px;
position: absolute;
cursor: pointer;
border: 1px solid blue;
z-index: 10;
margin-left: -6px;
left: 50%; }
.artifactop-decorator .connector:hover {
border-color: rgba(82, 168, 236, 0.8);
-webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
-moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6); }
.artifactop-decorator .connector.top {
top: -6px; }
.artifactop-decorator .connector.bottom {
bottom: -6px; }
.selected .artifactop-decorator {
border: 1px solid #52a8ec;
background-color: #dbeafc; }
@@ -0,0 +1,112 @@
/*globals define, _*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'js/Constants',
'decorators/DcOpDecorator/EasyDAG/DcOpDecorator.EasyDAGWidget',
'css!./ArtifactOpDecorator.EasyDAGWidget.css'
], function (
CONSTANTS,
DecoratorBase
) {
'use strict';
var ArtifactOpDecorator,
DECORATOR_ID = 'ArtifactOpDecorator',
CAST_OPTS = {
ArtifactLoader: {
ptr: 'artifact',
metaTgt: false
},
ArtifactFinder: {
ptr: 'type',
metaTgt: true
}
};
// ArtifactOp nodes need to be able to...
// - dynamically change their outputs (downcast)
ArtifactOpDecorator = function (options) {
options.color = options.color || '#b0bec5';
DecoratorBase.call(this, options);
// set the opts...
this.castOpts = CAST_OPTS[this._node.baseName];
};
_.extend(ArtifactOpDecorator.prototype, DecoratorBase.prototype);
ArtifactOpDecorator.prototype.DECORATOR_ID = DECORATOR_ID;
ArtifactOpDecorator.prototype.getTargetFilterFnFor = function() {
return id => {
var node = this.client.getNode(id),
isMetaTgt = node.getId() === node.getMetaTypeId();
return isMetaTgt === this.castOpts.metaTgt;
};
};
ArtifactOpDecorator.prototype.savePointer = function(name, to) {
// When the 'artifact' pointer is changed, we should change the base
// of the data output node to the target type
if (typeof to !== 'string') {
var outputId = this._node.outputs[0] && this._node.outputs[0].id;
// Clear the data handle of the output
this.client.startTransaction(`Removing output of ${this.name}`);
this.client.delPointer(this._node.id, name);
if (outputId) {
this.client.delAttributes(outputId, 'data');
}
this.client.completeTransaction();
} else if (name === this.castOpts.ptr) { // set the casted value
this.client.startTransaction(`Setting output of ${this.name} to ${to}`);
this.castOutputType(to);
this.client.makePointer(this._node.id, name, to);
this.client.completeTransaction();
} else {
DecoratorBase.prototype.savePointer.call(this, name, to);
}
};
ArtifactOpDecorator.prototype.getDisplayName = function() {
var ptrName = this._node.baseName === 'ArtifactLoader' ? 'artifact' : 'type',
id = this._node.pointers[ptrName],
name = this.nameFor[id] || this._node.name;
return name;
};
ArtifactOpDecorator.prototype.updateDisplayName = function() {
var newName = this.getDisplayName();
if (this.name !== newName) {
this.name = newName;
this.nameWidth = null;
}
};
ArtifactOpDecorator.prototype.updateTargetName = function(id, name) {
DecoratorBase.prototype.updateTargetName.apply(this, arguments);
// Update name
var ptrName = this._node.baseName === 'ArtifactLoader' ? 'artifact' : 'type';
if (this._node.pointers[ptrName] === id) {
this._name = name;
this.onResize();
}
};
ArtifactOpDecorator.prototype.expand = function() {
this.updateDisplayName();
DecoratorBase.prototype.expand.call(this);
};
ArtifactOpDecorator.prototype.condense = function() {
this.updateDisplayName();
DecoratorBase.prototype.condense.call(this);
};
return ArtifactOpDecorator;
});
@@ -0,0 +1,43 @@
/*globals define, _*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'js/Decorators/DecoratorBase',
'./EasyDAG/DcOpDecorator.EasyDAGWidget'
], function (
DecoratorBase,
DcOpDecoratorEasyDAGWidget
) {
'use strict';
var DcOpDecorator,
__parent__ = DecoratorBase,
__parent_proto__ = DecoratorBase.prototype,
DECORATOR_ID = 'DcOpDecorator';
DcOpDecorator = function (params) {
var opts = _.extend({loggerName: this.DECORATORID}, params);
__parent__.apply(this, [opts]);
this.logger.debug('DcOpDecorator ctor');
};
_.extend(DcOpDecorator.prototype, __parent_proto__);
DcOpDecorator.prototype.DECORATORID = DECORATOR_ID;
/*********************** OVERRIDE DecoratorBase MEMBERS **************************/
DcOpDecorator.prototype.initializeSupportedWidgetMap = function () {
this.supportedWidgetMap = {
EasyDAG: DcOpDecoratorEasyDAGWidget
};
};
return DcOpDecorator;
});
@@ -0,0 +1,43 @@
/**
* @author brollb / https://github.com/brollb
*/
.dcop-decorator {
min-width: 65px;
height: 40px;
border: 1px solid black;
background-color: #dedede;
padding: 3px;
text-align: center; }
.dcop-decorator .attr-title {
font-style: italic;
}
.dcop-decorator .name {
margin-top: 10px;
white-space: nowrap;
font-size: 16px;
font-weight: bold;
font-family: "Helvetica Neue",Helvetica,Arial,sans-serif;
text-align: center; }
.dcop-decorator .connector {
background-color: #fefefe;
height: 10px;
width: 10px;
position: absolute;
cursor: pointer;
border: 1px solid blue;
z-index: 10;
margin-left: -6px;
left: 50%; }
.dcop-decorator .connector:hover {
border-color: rgba(82, 168, 236, 0.8);
-webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
-moz-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 8px rgba(82, 168, 236, 0.6); }
.dcop-decorator .connector.top {
top: -6px; }
.dcop-decorator .connector.bottom {
bottom: -6px; }
.selected .dcop-decorator {
border: 1px solid #52a8ec;
background-color: #dbeafc; }
@@ -0,0 +1,79 @@
/*globals define, _*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'js/Constants',
'decorators/OperationDecorator/EasyDAG/OperationDecorator.EasyDAGWidget',
'css!./DcOpDecorator.EasyDAGWidget.css'
], function (
CONSTANTS,
DecoratorBase
) {
'use strict';
var DcOpDecorator,
DECORATOR_ID = 'DcOpDecorator';
// DcOp nodes need to be able to...
// - dynamically change their outputs (downcast)
DcOpDecorator = function (options) {
options.color = options.color || '#78909c';
DecoratorBase.call(this, options);
};
_.extend(DcOpDecorator.prototype, DecoratorBase.prototype);
DcOpDecorator.prototype.DECORATOR_ID = DECORATOR_ID;
DcOpDecorator.prototype.getTargetFilterFnFor = function() {
return id => {
var node = this.client.getNode(id);
return node.getId() !== node.getMetaTypeId(); // not meta node
};
};
DcOpDecorator.prototype.castOutputType = function(targetId) {
var target = this.client.getNode(targetId),
baseId = target.getBaseId(),
outputId = this._node.outputs[0] && this._node.outputs[0].id,
hash;
if (!outputId) {
// create the outputId node
outputId = this._createOutputNode(baseId);
} else {
this.client.makePointer(outputId, CONSTANTS.POINTER_BASE, baseId);
}
// Copy the data content to the output node
hash = target.getAttribute('data');
this.client.setAttributes(outputId, 'data', hash);
};
DcOpDecorator.prototype._createOutputNode = function(baseId) {
var n = this.client.getNode(this._node.id),
outputCntrId;
outputCntrId = n.getChildrenIds().find(id => {
var metaTypeId = this.client.getNode(id).getMetaTypeId(),
metaType = this.client.getNode(metaTypeId);
if (!metaType) {
this.logger.error(`Could not check the type of ${id}!`);
return false;
}
return metaType.getAttribute('name') === 'Outputs';
});
return this.client.createChild({
baseId: baseId,
parentId: outputCntrId
});
};
return DcOpDecorator;
});
@@ -18,6 +18,7 @@ define([
DECORATOR_ID = 'JobDecorator',
COLORS = {
pending: '#9e9e9e',
queued: '#cfd8dc',
running: '#fff59d',
success: '#66bb6a',
fail: '#e57373'
@@ -29,6 +30,13 @@ define([
// - unhighlight ports
// - report the location of specific ports
JobDecorator = function (options) {
options.skipAttributes = {
name: true,
status: true,
execFiles: true,
stdout: true,
debug: true
};
EllipseDecorator.call(this, options);
};
@@ -42,7 +50,8 @@ define([
JobDecorator.prototype.setAttributes = function() {
EllipseDecorator.prototype.setAttributes.call(this);
var status = this._attributes.status && this._attributes.status.value;
var attrs = this._node.attributes,
status = attrs.status && attrs.status.value;
// Update the color based on the 'status' attr
this.color = COLORS[status] || COLORS.fail;
@@ -0,0 +1,16 @@
/* globals define, _*/
define([
'decorators/EllipseDecorator/EasyDAG/AttributeField'
], function(
AttributeFieldBase
) {
// Attribute field in which the label is clickable and the attribute meta is editable
var AttributeField = function() {
AttributeFieldBase.apply(this, arguments);
this.$label.on('click', () => this.onLabelClick());
};
_.extend(AttributeField.prototype, AttributeFieldBase.prototype);
return AttributeField;
});
@@ -0,0 +1,26 @@
/* globals define */
define([
], function(
) {
var CreateAttrField = function(logger, pEl, y) {
this.$el = pEl.append('text')
.attr('y', y)
.attr('class', 'create-attr-field')
.attr('text-anchor', 'middle')
.attr('dominant-baseline', 'middle')
.attr('font-weight', 'bold')
.attr('font-style', 'italic')
.text('New Attribute')
.on('click', () => this.onClick());
};
CreateAttrField.prototype.render =
CreateAttrField.prototype.destroy = function() {};
CreateAttrField.prototype.width = function() {
return this.$el[0][0].getBoundingClientRect().width;
};
return CreateAttrField;
});
@@ -1,15 +1,17 @@
/*globals define, $,_*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'decorators/EllipseDecorator/EasyDAG/EllipseDecorator.EasyDAGWidget',
'./AttributeField',
'./CreateAttributeField',
'decorators/MetaDecorator/DiagramDesigner/AttributeDetailsDialog',
'css!./OpIntDecorator.EasyDAGWidget.css'
], function (
DecoratorBase
DecoratorBase,
AttributeField,
CreateAttributeField,
AttributeDetailsDialog
) {
'use strict';
@@ -17,13 +19,7 @@ define([
var OpIntDecorator,
DECORATOR_ID = 'OpIntDecorator';
// OpInt nodes need to be able to...
// - show their ports
// - highlight ports
// - unhighlight ports
// - report the location of specific ports
OpIntDecorator = function (options) {
this.color = this.color || '#78909c';
DecoratorBase.call(this, options);
};
@@ -31,9 +27,9 @@ define([
OpIntDecorator.prototype.DECORATOR_ID = DECORATOR_ID;
OpIntDecorator.prototype.initialize = function() {
if (this._node.baseName === 'Operation') {
if (this.isOperation()) {
this.color = '#2196f3';
} else {
} else if (this._node.baseName) {
// On hover, show the type
this.enableTooltip(this._node.baseName, 'dark');
}
@@ -41,6 +37,108 @@ define([
this.$name.on('dblclick', this.editName.bind(this));
};
OpIntDecorator.prototype.AttributeField = AttributeField;
OpIntDecorator.prototype.isOperation = function() {
return this._node.baseName === 'Operation';
};
OpIntDecorator.prototype.createAttributeFields = function(y, width) {
var field,
initialY = y;
if (!this.isOperation()) {
return y;
}
y = DecoratorBase.prototype.createAttributeFields.call(this, y, width);
// Change attribute field so clicking allows user to edit/delete the field
this.fields.forEach(field =>
field.onLabelClick = this.editAttributeMeta.bind(this, field.name));
// Add the 'create new attribute' field
y += this.ROW_HEIGHT + (y === initialY ? 0 : 10);
field = new CreateAttributeField(this.logger, this.$attributes, y, width);
field.onClick = this.newAttribute.bind(this);
this.fields.push(field);
return y;
};
OpIntDecorator.prototype.newAttribute = function() {
var defSchema = {
type: 'string'
};
this.editAttributeMeta(null, defSchema);
};
OpIntDecorator.prototype.expand = function() {
DecoratorBase.prototype.expand.call(this, this.isOperation());
};
OpIntDecorator.prototype.editAttributeMeta = function(name, defSchema) {
var dialog = new AttributeDetailsDialog(),
attrNames = Object.keys(this._node.attributes),
attrInfo = this._node.attributes[name] || defSchema,
schema,
i;
// Open the dialog for editing the attribute
schema = _.extend({defaultValue: attrInfo.value}, attrInfo);
// Remove the current name
i = attrNames.indexOf(name);
if (i !== -1) {
attrNames.splice(i, 1);
}
dialog.show(schema, attrNames,
desc => this.setAttributeMeta(name, desc),
() => this.deleteAttribute(name));
};
OpIntDecorator.prototype.deleteAttribute = function(name) {
var opName = this._node.attributes.name.value,
msg = `Deleting "${name}" attribute from "${opName}" operation`;
this.client.startTransaction(msg);
this.client.removeAttributeSchema(this._node.id, name);
this.client.delAttributes(this._node.id, name);
this.client.completeTransaction();
};
OpIntDecorator.prototype.setAttributeMeta = function(name, desc) {
var schema,
opName = this._node.attributes.name.value,
msg = `Updating "${name}" attribute in "${opName}" operation`;
// Create the schema from the desc
schema = {
type: desc.type,
min: desc.min,
max: desc.max,
regexp: desc.regexp
};
if (desc.isEnum) {
schema.enum = desc.enumValues;
}
// Update the operation's attribute
this.client.startTransaction(msg);
if (name !== desc.name) { // Renaming attribute
if (name) {
this.client.removeAttributeSchema(this._node.id, name);
this.client.delAttributes(this._node.id, name);
}
name = desc.name;
}
this.client.setAttributeSchema(this._node.id, name, schema);
this.client.setAttributes(this._node.id, name, desc.defaultValue);
this.client.completeTransaction();
};
OpIntDecorator.prototype.editName = function() {
var html = this.$name[0][0],
position = html.getBoundingClientRect(),
@@ -76,6 +174,13 @@ define([
};
OpIntDecorator.prototype.onNameChanged = function(oldVal, newValue) {
var whitespace = /^\s*$/;
if (newValue !== oldVal && !whitespace.test(newValue)) {
this.onValidNameChange(newValue);
}
};
OpIntDecorator.prototype.onValidNameChange = function(newValue) {
this.saveAttribute('name', newValue);
};
@@ -83,8 +188,5 @@ define([
return this._node.name;
};
// clicking on the name should allow the user to edit it in place
// TODO
return OpIntDecorator;
});
@@ -31,7 +31,7 @@ define([
OpIntPtrDecorator.prototype.DECORATOR_ID = DECORATOR_ID;
OpIntPtrDecorator.prototype.onNameChanged = function(old, newValue) {
OpIntPtrDecorator.prototype.onValidNameChange = function(newValue) {
return this.changePtrName(this.name, newValue);
};
@@ -1,10 +1,6 @@
/*globals define, _*/
/*globals define, _, Opentip*/
/*jshint browser: true, camelcase: false*/
/**
* @author brollb / https://github.com/brollb
*/
define([
'decorators/EllipseDecorator/EasyDAG/EllipseDecorator.EasyDAGWidget',
'css!./OperationDecorator.EasyDAGWidget.css'
@@ -16,7 +12,12 @@ define([
var OperationDecorator,
NAME_MARGIN = 25,
DECORATOR_ID = 'OperationDecorator';
DECORATOR_ID = 'OperationDecorator',
PORT_TOOLTIP_OPTS = {
tipJoint: 'left',
removeElementsOnHide: true,
style: 'dark'
};
// Operation nodes need to be able to...
// - show their ports
@@ -30,6 +31,7 @@ define([
this.id = this._node.id;
this.$ports = this.$el.append('g')
.attr('id', 'ports');
this.$portTooltips = {};
};
_.extend(OperationDecorator.prototype, DecoratorBase.prototype);
@@ -41,29 +43,22 @@ define([
};
OperationDecorator.prototype.condense = function() {
var path,
width,
rx;
width = Math.max(this.nameWidth + 2 * NAME_MARGIN, this.dense.width);
rx = width/2;
path = [
`M${-rx},0`,
`l ${width} 0`,
`l 0 ${this.dense.height}`,
`l -${width} 0`,
`l 0 -${this.dense.height}`
].join(' ');
var width = Math.max(this.nameWidth + 2 * NAME_MARGIN, this.dense.width);
this.$body
.attr('d', path);
.transition()
.attr('x', -width/2)
.attr('y', 0)
.attr('width', width)
.attr('height', this.dense.height);
// Clear the attributes
this.$attributes.remove();
this.clearFields();
this.$attributes = this.$el.append('g')
.attr('fill', '#222222');
.attr('fill', 'none');
this.createAttributeFields(0, width);
this.createPointerFields(0, width, true);
this.height = this.dense.height;
this.width = width;
@@ -91,7 +86,8 @@ define([
OperationDecorator.prototype.renderPort = function(port, x, y, isInput) {
var color = this.PORT_COLOR.OPEN,
portIcon = this.$ports.append('g');
portIcon = this.$ports.append('g'),
tooltip;
// If the port is incoming and occupied, render it differently
if (isInput && port.connection) {
@@ -115,14 +111,26 @@ define([
portIcon.on('click', this.onPortClick.bind(this, this.id, port.id, !isInput));
// Add tooltip with whole name
// TODO
if (this.$portTooltips[port.id]) {
this.$portTooltips[port.id].hide();
}
tooltip = new Opentip(portIcon[0][0], PORT_TOOLTIP_OPTS);
tooltip.setContent(port.name);
portIcon.on('mouseenter', () => tooltip.show());
portIcon.on('mouseout', () => tooltip.hide());
this.$portTooltips[port.id] = tooltip;
};
OperationDecorator.prototype.hidePorts = function() {
var visiblePortIds = Object.keys(this.$portTooltips);
this.logger.info(`hiding ports for ${this.name} (${this.id})`);
this.$ports.remove();
this.$ports = this.$el.append('g')
.attr('id', 'ports');
for (var i = visiblePortIds.length; i--;) {
this.$portTooltips[visiblePortIds[i]].hide();
}
};
OperationDecorator.prototype.getPortLocation = function(id, isInput) {
+169 -14
Ver Arquivo
@@ -1,19 +1,17 @@
/*globals define*/
/*jshint node:true, browser:true*/
/**
* Generated by PluginGenerator 1.7.0 from webgme on Mon May 23 2016 14:23:16 GMT-0500 (CDT).
* A plugin that inherits from the PluginBase. To see source code documentation about available
* properties and methods visit %host%/docs/source/PluginBase.html.
*/
define([
'q',
'deepforge/plugin/LocalExecutor',
'text!./metadata.json',
'underscore',
'plugin/PluginBase'
], function (
Q,
LocalExecutor,
pluginMetadata,
_,
PluginBase
) {
'use strict';
@@ -97,19 +95,44 @@ define([
return this.getExecutionDir()
.then(execDir => {
tgtNode = this.core.createNode({
base: this.META.Execution,
parent: execDir
});
this.core.setAttribute(tgtNode, 'name', `${name} Execution`);
return this.core.loadChildren(node);
// Get a unique name
return this.getUniqueExecName(name + '_execution');
})
.then(execName => {
var isSnapshot = this.getCurrentConfig().snapshot;
this.logger.debug(`Creating execution ${execName}`);
// Set all the metadata for the new execution
this.core.setAttribute(tgtNode, 'name', execName);
this.core.setAttribute(tgtNode, 'snapshot', isSnapshot);
this.core.setAttribute(tgtNode, 'tagname', execName);
this.core.setAttribute(tgtNode, 'createdAt', Date.now());
this.core.setPointer(tgtNode, 'origin', this.activeNode);
this.core.addMember(this.activeNode, 'executions', tgtNode);
return this.project.createTag(
execName.replace(/[ -]/g, '_'),
this.currentHash
);
})
.then(() => this.core.loadChildren(node))
.then(children => {
if (!children.length) {
this.logger.warn(`No children in pipeline. Will proceed anyway`);
this.logger.warn('No children in pipeline. Will proceed anyway');
}
copies = children.length ? this.core.copyNodes(children, tgtNode) : [];
return this.copyOperations(children, tgtNode);
})
.then(copiedPairs => {
var originals = copiedPairs.map(pair => pair[0]);
copies = copiedPairs.map(pair => pair[1]);
opTuples = copies
.map((copy, i) => [copy, i]) // zip w/ index
.filter(pair => this.core.isTypeOf(pair[0], this.META.Operation));
@@ -117,20 +140,147 @@ define([
// Create a mapping of old names to new names
return Q.all(opTuples.map(pair =>
// Add the input/output mappings to the dataMapping
this.addDataToMap(children[pair[1]], pair[0], dataMapping)
this.addDataToMap(originals[pair[1]], pair[0], dataMapping)
)
);
})
.then(() => { // datamapping is set!
this.updateReferences(copies, dataMapping);
this.boxOperations(opTuples.map(o => o[0]), tgtNode);
return this.save(`Created execution of ${name}`);
return this.save(`Created execution from ${name}`);
})
.then(() => tgtNode); // return tgtNode
};
CreateExecution.prototype.getExecutionsDir = function () {
return this.rootNode;
CreateExecution.prototype.getUniqueExecName = function (basename) {
var name = basename,
taken = {},
i = 2;
// Get a unique name wrt the tags and the other executions
return this.project.getTags()
.then(tags => {
Object.keys(tags).forEach(name => taken[name] = true);
// Get the other executions
return this.getExecutionDir();
})
.then(execDir => {
var cIds = this.core.getChildrenPaths(execDir);
return Q.all(cIds.map(id => this.core.loadByPath(this.rootNode, id)));
})
.then(execs => {
var names = execs.map(exec => this.core.getAttribute(exec, 'name'));
names.forEach(name => taken[name] = true);
while (taken[name]) {
name = basename + '_' + (i++);
}
return name;
});
};
CreateExecution.prototype.copyOperations = function (nodes, dst) {
var snapshot = this.getCurrentConfig().snapshot;
if (snapshot) {
return Q.all(nodes.map(node => {
if (this.isLocalOperation(node) ||
this.isMetaTypeOf(node, this.META.Transporter)) {
return [[node, this.core.copyNode(node, dst)]];
} else if (this.isMetaTypeOf(node, this.META.Operation)) {
return this.snapshotNode(node, dst);
}
}))
.then(pairs => pairs.filter(pair => !!pair)
.reduce((l1, l2) => l1.concat(l2))
);
} else if (nodes.length) {
var copies = this.core.copyNodes(nodes, dst);
return nodes.map((node, i) => [node, copies[i]]);
}
return [];
};
CreateExecution.prototype.snapshotNode = function (op, dst) {
// If we are making a snapshot, we should copy the base operation
// and set the attributes, add the child nodes, etc
var base = this.core.getBase(this.core.getBase(op)),
names,
values,
snapshot = this.core.createNode({
base: base,
parent: dst
});
// Copy over the attributes
names = this.core.getValidAttributeNames(op);
values = names.map(name => this.core.getAttribute(op, name));
names.forEach((name, i) =>
this.core.setAttribute(snapshot, name, values[i]));
// Copy the pointers
names = this.core.getValidPointerNames(op);
return Q.all(names
.map(name => this.core.getPointerPath(op, name))
.map(id => this.core.loadByPath(this.rootNode, id)))
.then(values => {
names.forEach((name, i) =>
this.core.setPointer(snapshot, name, values[i]));
// Copy the data I/O
var srcCntrs = this.core.getChildrenPaths(op),
dstCntrs = this.core.getChildrenPaths(snapshot);
return Q.all([srcCntrs, dstCntrs].map(ids =>
Q.all(ids.map(id => this.core.loadByPath(this.rootNode, id)))));
})
.then(cntrs => {
var srcCntrs,
dstCntrs;
// Sort all containers by metatype id
cntrs.map(l => l.sort((a, b) => {
var aId = this.core.getPath(this.core.getMetaType(a)),
bId = this.core.getPath(this.core.getMetaType(b));
return aId < bId ? -1 : 1;
}));
srcCntrs = cntrs[0];
dstCntrs = cntrs[1];
return Q.all(srcCntrs.map(ctr => Q.all(this.core.getChildrenPaths(ctr)
.map(id => this.core.loadByPath(this.rootNode, id)))))
.then(cntrs =>
cntrs.map((nodes, i) =>
nodes.map(n => [n, this.copyDataNode(n, dstCntrs[i])]))
);
})
.then(nodes => {
nodes = nodes.reduce((l1, l2) => l1.concat(l2), []);
nodes.push([op, snapshot]);
return nodes;
});
};
CreateExecution.prototype.copyDataNode = function (original, dst) {
// Create new node of the given type
var attrNames = this.core.getAttributeNames(original),
values,
copy = this.core.createNode({
base: this.core.getMetaType(original),
parent: dst
});
// Set the 'name', 'data' attributes
values = attrNames.map(name => this.core.getAttribute(original, name));
attrNames.forEach((name, i) =>
this.core.setAttribute(copy, name, values[i]));
return copy;
};
CreateExecution.prototype.addDataToMap = function (srcOp, dstOp, map) {
@@ -209,5 +359,10 @@ define([
});
};
_.extend(
CreateExecution.prototype,
LocalExecutor.prototype
);
return CreateExecution;
});
+12 -3
Ver Arquivo
@@ -1,6 +1,6 @@
{
"id": "CreateExecution",
"name": "CreateExecution",
"name": "Create Execution",
"version": "0.1.0",
"description": "",
"icon": {
@@ -10,5 +10,14 @@
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"writeAccessRequired": false,
"configStructure": []
}
"configStructure": [
{
"name": "snapshot",
"displayName": "Snapshot",
"description": "Freeze the operation definitions and attributes at current value",
"value": true,
"valueType": "boolean",
"readOnly": false
}
]
}
+190 -47
Ver Arquivo
@@ -9,12 +9,22 @@ define([
'plugin/PluginConfig',
'plugin/PluginBase',
'deepforge/js-yaml.min',
'text!deepforge/layers.yml',
'common/util/guid',
'js/RegistryKeys',
'js/Constants',
'js/Panels/MetaEditor/MetaEditorConstants',
'underscore',
'text!deepforge/layers.json',
'text!./metadata.json'
], function (
PluginConfig,
PluginBase,
yaml,
generateGuid,
REGISTRY_KEYS,
CONSTANTS,
META_CONSTANTS,
_,
DEFAULT_LAYERS,
metadata
) {
@@ -31,6 +41,8 @@ define([
// Call base class' constructor.
PluginBase.call(this);
this.pluginMetadata = CreateTorchMeta.metadata;
this.metaSheets = {};
this.sheetCounts = {};
};
CreateTorchMeta.metadata = JSON.parse(metadata);
@@ -58,7 +70,7 @@ define([
}
// Extra layer names
this.getYamlText((err, text) => {
this.getJsonLayers((err, text) => {
if (err) {
return callback(err, this.result);
}
@@ -67,15 +79,24 @@ define([
// - (Abstract) CategoryLayerTypes
// - LayerName
// - Attributes (if exists)
var content,
var content = {},
categories,
nodes = {};
config = this.getCurrentConfig(),
nodes = {},
layers;
try {
content = yaml.load(text);
layers = JSON.parse(text);
} catch (e) {
return callback('YAML parse error: ' + e, this.result);
return callback('JSON parse error: ' + e, this.result);
}
layers.forEach(layer => {
if (!content[layer.type]) {
content[layer.type] = [];
}
content[layer.type].push(layer);
});
categories = Object.keys(content);
// Create the base class, if needed
if (!this.META.Layer) {
@@ -84,23 +105,52 @@ define([
// Create the category nodes
categories
.forEach(name => nodes[name] = this.createMetaNode(name, this.META.Layer));
.forEach(name => {
// Create a tab for each
this.metaSheets[name] = this.createMetaSheetTab(name);
this.sheetCounts[name] = 0;
nodes[name] = this.createMetaNode(name, this.META.Layer, name);
});
// Make them abstract
categories
.forEach(name => this.core.setRegistry(nodes[name], 'isAbstract', true));
if (config.removeOldLayers) {
var isNewLayer = {},
newLayers = layers.map(layer => layer.name),
oldLayers,
oldNames;
newLayers = newLayers.concat(categories); // add the category nodes
newLayers.forEach(name => isNewLayer[name] = true);
// Set the newLayer nodes 'base' to 'Layer' so we don't accidentally
// delete them
newLayers
.map(name => this.META[name])
.filter(layer => !!layer)
.forEach(layer => this.core.setPointer(layer, 'base', this.META.Layer));
oldLayers = Object.keys(this.META)
.filter(name => name !== 'Layer')
.map(name => this.META[name])
.filter(node => this.isMetaTypeOf(node, this.META.Layer))
.filter(node => !isNewLayer[this.core.getAttribute(node, 'name')]);
oldNames = oldLayers.map(l => this.core.getAttribute(l, 'name'));
// Get the old layer names
this.logger.debug(`Removing layers: ${oldNames.join(', ')}`);
oldLayers.forEach(layer => this.core.deleteNode(layer));
}
// Create the actual nodes
categories.forEach(cat => {
content[cat]
.forEach(name => {
var attrs = null;
if (typeof name !== 'string') {
attrs = name[Object.keys(name)[0]];
name = Object.keys(name)[0];
}
nodes[name] = this.createMetaNode(name, nodes[cat], attrs);
.forEach(layer => {
var attrs = layer.params,
name = layer.name;
nodes[name] = this.createMetaNode(name, nodes[cat], cat, attrs);
// Make the node non-abstract
this.core.setRegistry(nodes[name], 'isAbstract', false);
});
@@ -115,11 +165,48 @@ define([
callback(null, self.result);
});
});
};
CreateTorchMeta.prototype.getYamlText = function (callback) {
CreateTorchMeta.prototype.removeFromMeta = function (nodeId) {
var sheets = this.core.getRegistry(this.rootNode, REGISTRY_KEYS.META_SHEETS),
sheet;
// Remove from meta
this.core.delMember(this.rootNode, META_CONSTANTS.META_ASPECT_SET_NAME, nodeId);
// Remove from the given meta sheet
sheet = sheets.find(sheet => {
var paths = this.core.getMemberPaths(this.rootNode, sheet.SetID);
return paths.indexOf(nodeId) > -1;
});
if (sheet) {
this.core.delMember(this.rootNode, sheet.SetID, nodeId);
}
};
CreateTorchMeta.prototype.createMetaSheetTab = function (name) {
var sheets = this.core.getRegistry(this.rootNode, REGISTRY_KEYS.META_SHEETS),
id = META_CONSTANTS.META_ASPECT_SHEET_NAME_PREFIX + generateGuid(),
sheet,
desc = {
SetID: id,
order: sheets.length,
title: name
};
sheet = sheets.find(sheet => sheet.title === name);
if (!sheet) {
sheet = desc;
this.logger.debug(`creating meta sheet "${name}"`);
this.core.createSet(this.rootNode, sheet.SetID);
sheets.push(sheet);
this.core.setRegistry(this.rootNode, REGISTRY_KEYS.META_SHEETS, sheets);
}
return sheet.SetID;
};
CreateTorchMeta.prototype.getJsonLayers = function (callback) {
var config = this.getCurrentConfig();
if (config.layerNameHash) {
@@ -135,41 +222,72 @@ define([
}
};
CreateTorchMeta.prototype.createMetaNode = function (name, base, attrs) {
var node;
CreateTorchMeta.prototype.createMetaNode = function (name, base, tabName, attrs) {
var node = this.META[name],
nodeId = node && this.core.getPath(node),
tabId = this.metaSheets[tabName],
position = this.getPositionFor(name, tabName);
if (this.META[name]) {
this.logger.warn('"' + name + '" already exists. skipping...');
return this.META[name];
if (!tabId) {
this.logger.error(`No meta sheet for ${tabName}`);
}
// Create a node
node = this.core.createNode({
parent: this.META.Language,
base: base
});
this.core.setAttribute(node, 'name', name);
if (!node) {
// Create a node
node = this.core.createNode({
parent: this.META.Language,
base: base
});
this.core.setAttribute(node, 'name', name);
nodeId = this.core.getPath(node);
} else {
// Remove from meta
this.removeFromMeta(nodeId);
this.core.setPointer(node, 'base', base);
}
// Add it to the meta sheet
this.core.addMember(this.rootNode, 'MetaAspectSet', node);
this.core.addMember(this.rootNode, META_CONSTANTS.META_ASPECT_SET_NAME, node);
this.core.addMember(this.rootNode, tabId, node);
// Add it to a tab of the meta sheet
var set = this.core.getSetNames(this.rootNode)
.find(name => name !== 'MetaAspectSet');
this.core.addMember(this.rootNode, set, node);
// TODO: Position the nodes on the META
// TODO: Put each group of nodes on their own META sheet
this.core.setMemberRegistry(
this.rootNode,
META_CONSTANTS.META_ASPECT_SET_NAME,
nodeId,
REGISTRY_KEYS.POSITION,
position
);
this.core.setMemberRegistry(
this.rootNode,
tabId,
nodeId,
REGISTRY_KEYS.POSITION,
position
);
if (attrs) { // Add the attributes
attrs.forEach((name, index) => {
var desc = null;
if (typeof name !== 'string') {
desc = name[Object.keys(name)[0]];
name = Object.keys(name)[0];
// Remove attributes not in the given list
var currentAttrs = this.core.getValidAttributeNames(node),
rmAttrs;
rmAttrs = _.difference(currentAttrs, attrs) // old attribute names
.filter(attr => attr !== 'name');
if (rmAttrs.length) {
this.logger.debug(`Removing ${rmAttrs.join(', ')} from ${name}`);
}
rmAttrs.forEach(attr => {
this.core.delAttributeMeta(node, attr);
if (this.core.getOwnAttribute(node, attr) !== undefined) {
this.core.delAttribute(node, attr);
}
desc = desc || {};
});
attrs.forEach((name, index) => {
var desc = {};
desc.argindex = index;
desc.default = '';
this.addAttribute(name, node, desc);
});
}
@@ -178,11 +296,38 @@ define([
return node;
};
CreateTorchMeta.prototype.getPositionFor = function(name, tabName) {
var index = this.sheetCounts[tabName],
dx = 140,
dy = 100,
MAX_WIDTH = 1200,
x;
if (tabName === 'Convolution') {
dx *= 1.3;
dy *= 1.5;
}
this.sheetCounts[tabName]++;
if (index === 0) {
return {
x: MAX_WIDTH/2,
y: 50
};
}
x = dx*index;
return {
x: x%MAX_WIDTH,
y: Math.floor(x/MAX_WIDTH+1)*dy + 50
};
};
CreateTorchMeta.prototype.addAttribute = function (name, node, def) {
var initial,
schema = {};
schema.type = def.type || 'integer';
schema.type = def.type || 'string';
if (schema.type === 'list') { // FIXME: add support for lists
schema.type = 'string';
}
@@ -212,9 +357,7 @@ define([
if (schema.type === 'boolean') {
initial = initial !== null ? initial : false;
}
if (initial !== null) { // optional attribute - set default value
this.core.setAttribute(node, name, initial);
}
this.core.setAttribute(node, name, initial);
};
return CreateTorchMeta;
+9 -1
Ver Arquivo
@@ -1,6 +1,6 @@
{
"id": "CreateTorchMeta",
"name": "Create Torch Meta",
"name": "Update nn meta",
"version": "0.1.0",
"description": "Create metamodel from Torch yaml",
"icon": {
@@ -17,6 +17,14 @@
"value": "",
"valueType": "asset",
"readOnly": false
},
{
"name": "removeOldLayers",
"displayName": "Delete old layers",
"description": "Delete all layers not in the current description",
"value": true,
"valueType": "boolean",
"readOnly": false
}
]
}
+405 -176
Ver Arquivo
@@ -1,25 +1,27 @@
/*globals define, WebGMEGlobal*/
/*globals define */
/*jshint node:true, browser:true, esversion: 6*/
define([
'plugin/CreateExecution/CreateExecution/CreateExecution',
'deepforge/plugin/PtrCodeGen',
'common/storage/constants',
'common/core/constants',
'q',
'text!./metadata.json',
'./Templates/index',
'./LocalExecutor',
'./templates/index',
'deepforge/plugin/LocalExecutor',
'executor/ExecutorClient',
'jszip',
'underscore'
], function (
CreateExecution,
PtrCodeGen,
STORAGE_CONSTANTS,
CONSTANTS,
Q,
pluginMetadata,
Templates,
LocalExecutor, // DeepForge operation primitives
ExecutorClient,
JsZip,
_
) {
'use strict';
@@ -32,24 +34,15 @@ define([
* @classdesc This class represents the plugin ExecutePipeline.
* @constructor
*/
var OUTPUT_INTERVAL = 1500,
STDOUT_FILE = 'job_stdout.txt';
var ExecutePipeline = function () {
// Call base class' constructor.
CreateExecution.call(this);
this.pluginMetadata = pluginMetadata;
// Cache
this.nodes = {};
// Record keeping for running operations
this.opFor = {};
this.incomingCounts = {};
this.outputsOf = {};
this.inputPortsFor = {};
this.inputs = {};
this.finished = {};
this.completedCount = 0;
this.totalCount = 0;
this._currentSave = Q();
this.initRun();
};
/**
@@ -64,6 +57,33 @@ define([
ExecutePipeline.prototype = Object.create(CreateExecution.prototype);
ExecutePipeline.prototype.constructor = ExecutePipeline;
ExecutePipeline.prototype.initRun = function () {
// Cache
this.nodes = {};
// Record keeping for running operations
this.opFor = {};
this.incomingCounts = {};
this.outputsOf = {};
this.inputPortsFor = {};
this.inputs = {};
this.finished = {};
this.completedCount = 0;
this.totalCount = 0;
this.outputLineCount = {};
// When a pipeline fails, it will let all running jobs finish and record
// the results of each job
//
// The following variables are used to...
// - keep track of the number of jobs currently running
// - keep track if the pipeline has errored
// - if so, don't start any more jobs
this.pipelineError = null;
this.runningJobs = 0;
};
/**
* Main function for the plugin to execute. This will perform the execution.
* Notes:
@@ -78,16 +98,16 @@ define([
// inputs for the next operation cannot be created until the inputs have
// been generated
this.initRun();
var startPromise;
if (this.core.isTypeOf(this.activeNode, this.META.Pipeline)) {
// If starting with a pipeline, we will create an Execution first
startPromise = this.createExecution(this.activeNode)
.then(execNode => {
this.activeNode = execNode;
return this.core.loadSubTree(this.activeNode);
});
} else if (this.core.isTypeOf(this.activeNode, this.META.Execution)) {
startPromise = this.core.loadSubTree(this.activeNode);
startPromise = Q();
} else {
return callback('Current node is not a Pipeline or Execution!', this.result);
}
@@ -96,41 +116,85 @@ define([
this.debug = true; // this.getCurrentConfig().debug;
this._callback = callback;
startPromise.then(subtree => {
startPromise
.then(() => this.core.loadSubTree(this.activeNode))
.then(subtree => {
var children = subtree
.filter(n => this.core.getParent(n) === this.activeNode);
this.pipelineName = this.core.getAttribute(this.activeNode, 'name');
this.buildCache(subtree);
this.parsePipeline(children); // record deps, etc
//if (this.getCurrentConfig().reset) {
this.clearResults();
//}
// Execute the operations in the proper order
this.executePipeline();
return this.clearResults();
})
.then(() => this.executePipeline())
.fail(e => this.logger.error(e));
};
ExecutePipeline.prototype.updateForkName = function () {
var basename = this.pipelineName + '_fork';
return this.project.getBranches().then(branches => {
var names = Object.keys(branches),
name = basename,
i = 2;
while (names.indexOf(name) !== -1) {
name = basename + '_' + i;
i++;
}
this.forkName = name;
});
};
// Override 'save' to prevent race conditions while saving
ExecutePipeline.prototype.save = function (msg) {
// When 'save' is called, it should still finish any current save op
// before continuing
this._currentSave = this._currentSave
.then(() => this.updateForkName())
.then(() => CreateExecution.prototype.save.call(this, msg))
.then(result => {
var msg;
if (result.status === STORAGE_CONSTANTS.FORKED) {
msg = `"${this.pipelineName}" execution has forked to "${result.forkName}"`;
this.sendNotification(msg);
}
});
return this._currentSave;
};
ExecutePipeline.prototype.isInputData = function (node) {
var prnt = this.core.getParent(node);
return this.core.isTypeOf(prnt, this.META.Inputs);
};
ExecutePipeline.prototype.clearResults = function () {
var nodes = Object.keys(this.nodes).map(id => this.nodes[id]);
// Clear the pipeline's results
this.logger.info('Clearing all intermediate execution results');
nodes.filter(node => this.core.isTypeOf(node, this.META.Data))
// Only input data nodes should be cleared. Outputs will be overwritten
.filter(node => this.isInputData(node))
.forEach(conn => this.core.delAttribute(conn, 'data'));
// Set the status for each job to 'pending'
nodes.filter(node => this.core.isTypeOf(node, this.META.Job))
.forEach(node => this.core.setAttribute(node, 'status', 'pending'));
// Set the status of the execution to 'running'
this.core.setAttribute(this.activeNode, 'status', 'running');
this.logger.info('Setting all jobs status to "pending"');
this.logger.debug(`Making a commit from ${this.currentHash}`);
return this.save(`Initializing ${this.pipelineName} for execution`);
};
//////////////////////////// Operation Preparation/Execution ////////////////////////////
ExecutePipeline.prototype.buildCache = function (nodes) {
// Cache all nodes
// Do I need to cache the data inputs? TODO
// Probably not - I should be able to look them up as needed
nodes.forEach(node => this.nodes[this.core.getPath(node)] = node);
};
@@ -216,12 +280,28 @@ define([
ExecutePipeline.prototype.onPipelineComplete = function(err) {
var name = this.core.getAttribute(this.activeNode, 'name');
this.logger.debug(`Pipeline "${name}" complete!`);
if (err) {
this.runningJobs--;
}
this.pipelineError = this.pipelineError || err;
if (this.pipelineError && this.runningJobs > 0) {
this.logger.info('Pipeline errored but is waiting for the running ' +
'jobs to finish');
return;
}
this.logger.debug(`Pipeline "${name}" complete!`);
this.core.setAttribute(this.activeNode, 'status',
(!this.pipelineError ? 'success' : 'failed'));
this._finished = true;
this.save('Pipeline execution finished')
.then(() => {
this.result.setSuccess(!err);
this._callback(err || null, this.result);
this.result.setSuccess(!this.pipelineError);
this._callback(this.pipelineError || null, this.result);
})
.fail(e => this.logger.error(e));
};
@@ -232,11 +312,25 @@ define([
readyOps = operations.filter(name => this.incomingCounts[name] === 0);
this.logger.info(`About to execute ${readyOps.length} operations`);
// If the pipeline has errored don't start any more jobs
if (this.pipelineError) {
if (this.runningJobs === 0) {
this.onPipelineComplete();
}
return 0;
}
// Execute all ready operations
readyOps.forEach(jobId => {
delete this.incomingCounts[jobId];
this.executeOperation(jobId);
});
readyOps.reduce((prev, jobId) => {
return prev.then(() => this.executeOperation(jobId));
}, Q());
this.runningJobs += readyOps.length;
this.logger.info(`There are now ${this.runningJobs} running jobs`);
return readyOps.length;
};
@@ -251,7 +345,7 @@ define([
ExecutePipeline.prototype.executeOperation = function (jobId) {
var node = this.getOperation(jobId),
name = this.core.getAttribute(node, 'name'),
localTypeId = this.getLocalOpType(node),
localTypeId = this.getLocalOperationType(node),
artifact,
artifactName,
files,
@@ -259,11 +353,13 @@ define([
inputs;
// Execute any special operation types here - not on an executor
this.logger.debug(`Executing operation "${name}"`);
if (localTypeId !== null) {
this.executeLocalOperation(localTypeId, node);
return this.executeLocalOperation(localTypeId, node);
} else {
// Generate all execution files
this.createOperationFiles(node).then(results => {
return this.createOperationFiles(node).then(results => {
this.logger.info('Created operation files!');
files = results;
artifactName = `${name}_${jobId.replace(/\//g, '_')}-execution-files`;
artifact = this.blobClient.createArtifact(artifactName);
@@ -306,8 +402,8 @@ define([
})
.then(outputArgs => {
var config,
args = ['init.lua'],
outputs;
outputs,
file;
outputs = outputArgs.map(pair => pair[0])
.map(name => {
@@ -317,8 +413,12 @@ define([
};
});
outputs.push({
name: 'stdout',
resultPatterns: [STDOUT_FILE]
});
if (this.debug) {
args.push('#' + Date.now());
outputs.push({
name: name + '-all-files',
resultPatterns: []
@@ -326,13 +426,22 @@ define([
}
config = {
cmd: 'th',
args: args,
cmd: 'bash',
args: ['run.sh'],
outputInterval: OUTPUT_INTERVAL,
resultArtifacts: outputs
};
files['executor_config.json'] = JSON.stringify(config, null, 4);
files['run.sh'] = Templates.BASH;
// Save the artifact
// Remove empty hashes
for (file in data) {
if (!data[file]) {
this.logger.warn(`Empty data hash has been found for file "${file}". Removing it...`);
delete data[file];
}
}
return artifact.addObjectHashes(data);
})
.then(() => {
@@ -350,6 +459,7 @@ define([
})
.fail(e => {
this.core.setAttribute(this.nodes[jobId], 'status', 'fail');
this.logger.info(`Setting ${jobId} status to "fail"`);
this.onPipelineComplete(`Distributed operation "${name}" failed ${e}`);
});
}
@@ -365,22 +475,57 @@ define([
this.logger.info(`Executing operation "${name}"`);
this.outputLineCount[jobId] = 0;
// Set the job status to 'running'
this.core.setAttribute(this.nodes[jobId], 'status', 'running');
// Run the operation on an executor
executor.createJob({hash})
this.core.setAttribute(this.nodes[jobId], 'status', 'queued');
this.core.setAttribute(this.nodes[jobId], 'stdout', '');
this.logger.info(`Setting ${jobId} status to "queued" (${this.currentHash})`);
this.logger.debug(`Making a commit from ${this.currentHash}`);
this.save(`Queued "${name}" operation in ${this.pipelineName}`)
.then(() => executor.createJob({hash}))
.then(() => this.watchOperation(executor, hash, opId, jobId))
.catch(err => this.logger.error(`Could not execute "${name}": ${err}`));
};
ExecutePipeline.prototype.watchOperation = function (executor, hash, opId, jobId) {
var name;
var job = this.nodes[jobId],
info,
name;
return executor.getInfo(hash)
.then(info => {
.then(_info => { // Update the job's stdout
var actualLine, // on executing job
currentLine = this.outputLineCount[jobId];
info = _info;
actualLine = info.outputNumber;
if (actualLine !== null && actualLine >= currentLine) {
this.outputLineCount[jobId] = actualLine + 1;
return executor.getOutput(hash, currentLine, actualLine+1)
.then(outputLines => {
var stdout = this.core.getAttribute(job, 'stdout'),
output = outputLines.map(o => o.output).join(''),
jobName = this.core.getAttribute(job, 'name');
// Handle the \b
// TODO
stdout += output;
this.core.setAttribute(job, 'stdout', stdout);
return this.save(`Received stdout for ${jobName}`);
});
}
})
.then(() => {
if (info.status === 'CREATED' || info.status === 'RUNNING') {
if (info.status === 'RUNNING' &&
this.core.getAttribute(this.nodes[jobId], 'status') !== 'running') {
name = this.core.getAttribute(this.nodes[jobId], 'name');
this.core.setAttribute(this.nodes[jobId], 'status', 'running');
this.save(`Started "${name}" operation in ${this.pipelineName}`);
}
setTimeout(
this.watchOperation.bind(this, executor, hash, opId, jobId),
ExecutePipeline.UPDATE_INTERVAL
@@ -388,20 +533,29 @@ define([
return;
}
if (info.status !== 'SUCCESS') {
name = this.core.getAttribute(this.nodes[opId], 'name');
// Download all files
this.result.addArtifact(info.resultHashes[name + '-all-files']);
// Set the job to failed! Store the error
this.core.setAttribute(this.nodes[jobId], 'status', 'fail');
this.onPipelineComplete(`Operation "${opId}" failed! ${JSON.stringify(info)}`); // Failed
} else {
name = this.core.getAttribute(this.nodes[opId], 'name');
if (this.debug) {
this.result.addArtifact(info.resultHashes[name + '-all-files']);
}
this.onDistOperationComplete(opId, info);
}
name = this.core.getAttribute(job, 'name');
this.core.setAttribute(job, 'execFiles', info.resultHashes[name + '-all-files']);
return this.blobClient.getArtifact(info.resultHashes.stdout)
.then(artifact => {
var stdoutHash = artifact.descriptor.content[STDOUT_FILE].content;
return this.blobClient.getObjectAsString(stdoutHash);
})
.then(stdout => {
this.core.setAttribute(job, 'stdout', stdout);
if (info.status !== 'SUCCESS') {
// Download all files
this.result.addArtifact(info.resultHashes[name + '-all-files']);
// Set the job to failed! Store the error
this.core.setAttribute(this.nodes[jobId], 'status', 'fail');
this.logger.info(`Setting ${jobId} status to "fail"`);
this.onPipelineComplete(`Operation "${opId}" failed! ${JSON.stringify(info)}`); // Failed
} else {
if (this.debug) {
this.result.addArtifact(info.resultHashes[name + '-all-files']);
}
this.onDistOperationComplete(opId, info);
}
});
})
.catch(err => this.logger.error(`Could not get op info for ${opId}: ${err}`));
};
@@ -430,9 +584,13 @@ define([
// Create new metadata for each
artifacts.forEach((artifact, i) => {
var name = outputs[i][0],
hash = artifact.descriptor.content[`outputs/${name}`].content;
outputData = artifact.descriptor.content[`outputs/${name}`],
hash = outputData && outputData.content;
this.core.setAttribute(outputMap[name], 'data', hash);
if (hash) {
this.core.setAttribute(outputMap[name], 'data', hash);
this.logger.info(`Setting ${nodeId} data to ${hash}`);
}
});
return this.onOperationComplete(node);
@@ -445,45 +603,55 @@ define([
nextPortIds = this.getOperationOutputIds(opNode),
jNode = this.core.getParent(opNode),
resultPorts,
jobId = this.core.getPath(jNode),
hasReadyOps;
// Set the operation to 'success'!
this.runningJobs--;
this.core.setAttribute(jNode, 'status', 'success');
this.logger.info(`Setting ${jobId} status to "success"`);
this.logger.info(`There are now ${this.runningJobs} running jobs`);
this.logger.debug(`Making a commit from ${this.currentHash}`);
this.save(`Operation "${name}" in ${this.pipelineName} completed successfully`)
.then(() => {
// Transport the data from the outputs to any connected inputs
// - Get all the connections from each outputId
// - Get the corresponding dst outputs
// - Use these new ids for checking 'hasReadyOps'
resultPorts = nextPortIds.map(id => this.inputPortsFor[id])
.reduce((l1, l2) => l1.concat(l2), []);
// Transport the data from the outputs to any connected inputs
// - Get all the connections from each outputId
// - Get the corresponding dst outputs
// - Use these new ids for checking 'hasReadyOps'
resultPorts = nextPortIds.map(id => this.inputPortsFor[id])
.reduce((l1, l2) => l1.concat(l2), []);
resultPorts.map((id, i) => [this.nodes[id], this.nodes[nextPortIds[i]]])
.forEach(pair => { // [ resultPort, nextPort ]
var result = pair[0],
next = pair[1],
hash = this.core.getAttribute(result, 'data');
this.logger.info(`forwarding data (${hash}) from ${this.core.getPath(result)} ` +
`to ${this.core.getPath(next)}`);
this.core.setAttribute(next, 'data', hash);
resultPorts
.map((id, i) => [this.nodes[id], this.nodes[nextPortIds[i]]])
.forEach(pair => { // [ resultPort, nextPort ]
var result = pair[0],
next = pair[1],
hash = this.core.getAttribute(result, 'data');
this.logger.info(`forwarding data (${hash}) from ${this.core.getPath(result)} ` +
`to ${this.core.getPath(next)}`);
this.core.setAttribute(next, 'data', hash);
this.logger.info(`Setting ${jobId} data to ${hash}`);
});
// For all the nextPortIds, decrement the corresponding operation's incoming counts
hasReadyOps = nextPortIds.map(id => this.getSiblingIdContaining(id))
.reduce((l1, l2) => l1.concat(l2), [])
// decrement the incoming counts for each operation id
.map(opId => --this.incomingCounts[opId])
.indexOf(0) > -1;
this.completedCount++;
this.logger.debug(`Operation "${name}" completed. ` +
`${this.totalCount - this.completedCount} remaining.`);
if (hasReadyOps) {
this.executeReadyOperations();
} else if (this.completedCount === this.totalCount) {
this.onPipelineComplete();
}
});
// For all the nextPortIds, decrement the corresponding operation's incoming counts
hasReadyOps = resultPorts.map(id => this.opFor[id])
.reduce((l1, l2) => l1.concat(l2), [])
// decrement the incoming counts for each operation id
.map(opId => --this.incomingCounts[opId])
.indexOf(0) > -1;
this.completedCount++;
this.logger.info(`Operation "${name}" completed. ` +
`${this.totalCount - this.completedCount} remaining.`);
if (hasReadyOps) {
this.executeReadyOperations();
} else if (this.completedCount === this.totalCount) {
this.onPipelineComplete();
}
};
ExecutePipeline.prototype.getOperationOutputIds = function(node) {
@@ -510,7 +678,10 @@ define([
// <name>.lua (entry point -> calls main operation code)
// add the given files
this.logger.info('About to create dist execution files');
return this.createEntryFile(node, files)
.then(() => this.createClasses(node, files))
.then(() => this.createCustomLayers(node, files))
.then(() => this.createInputs(node, files))
.then(() => this.createOutputs(node, files))
.then(() => this.createMainFile(node, files))
@@ -520,34 +691,120 @@ define([
});
};
ExecutePipeline.prototype.createClasses = function (node, files) {
var isClass = {},
metaDict = this.core.getAllMetaNodes(this.rootNode),
metanodes,
classNodes,
code;
this.logger.info('Creating custom layer file...');
metanodes = Object.keys(metaDict).map(id => metaDict[id]);
// Get all the custom layers
metanodes.forEach(node => {
if (this.core.getAttribute(node, 'name') === 'Complex') {
isClass[this.core.getPath(node)] = true;
}
});
classNodes = metanodes.filter(node => {
var base = this.core.getBase(node),
baseId = this.core.getPath(base);
return isClass[baseId];
});
// Get the code definitions for each
code = classNodes.map(node =>
`require './${this.core.getAttribute(node, 'name')}.lua'`
).join('\n');
// Create the class files
classNodes.forEach(node => {
var name = this.core.getAttribute(node, 'name');
files[`classes/${name}.lua`] = this.core.getAttribute(node, 'code');
});
// Create the custom layers file
files['classes/init.lua'] = code;
};
ExecutePipeline.prototype.createCustomLayers = function (node, files) {
var isCustomLayer = {},
metaDict = this.core.getAllMetaNodes(this.rootNode),
metanodes,
customLayers,
code;
this.logger.info('Creating custom layer file...');
metanodes = Object.keys(metaDict).map(id => metaDict[id]);
// Get all the custom layers
metanodes.forEach(node => {
if (this.core.getAttribute(node, 'name') === 'CustomLayer') {
isCustomLayer[this.core.getPath(node)] = true;
}
});
customLayers = metanodes.filter(node =>
this.core.getMixinPaths(node).some(id => isCustomLayer[id]));
// Get the code definitions for each
code = 'require \'nn\'\n\n' + customLayers
.map(node => this.core.getAttribute(node, 'code')).join('\n');
// Create the custom layers file
files['custom-layers.lua'] = code;
};
ExecutePipeline.prototype.createInputs = function (node, files) {
var tplContents;
this.logger.info('Retrieving inputs and deserialize fns...');
return this.getInputs(node)
.then(inputs => {
// For each input, match the connection with the input name
// [ name, type ] => [ name, type, node ]
if (inputs.length > 1) {
this.logger.warn('multiple inputs not yet fully supported!');
}
//
// For each input,
// - create the deserializer
// - put it in inputs/<name>/init.lua
// - copy the data asset to /inputs/<name>/init.lua
inputs = inputs
.filter(pair => !!this.core.getAttribute(pair[2], 'data')); // remove empty inputs
files.inputAssets = {}; // data assets
tplContents = inputs.map(pair => {
var name = pair[0],
node = pair[2];
node = pair[2],
nodeId = this.core.getPath(node),
fromNodeId,
fromNode,
deserFn,
base,
className;
// Get the deserialize function. First, try to get it from
// the source method (this guarantees that the correct
// deserialize method is used despite any auto-upcasting
fromNodeId = this.inputPortsFor[nodeId][0] || nodeId;
fromNode = this.nodes[fromNodeId];
deserFn = this.core.getAttribute(fromNode, 'deserialize');
if (this.isMetaTypeOf(node, this.META.Complex)) {
// Complex objects are expected to define their own
// (static) deserialize factory method
base = this.core.getBase(node);
className = this.core.getAttribute(base, 'name');
deserFn = `return ${className}.deserialize(path)`;
}
return {
name: name,
code: this.core.getAttribute(node, 'deserialize')
code: deserFn
};
});
var hashes = inputs.map(pair =>
var hashes = inputs
// storing the hash for now...
files.inputAssets[pair[0]] = this.core.getAttribute(pair[2], 'data')
);
.map(pair =>
files.inputAssets[pair[0]] = this.core.getAttribute(pair[2], 'data')
);
return Q.all(hashes.map(h => this.blobClient.getMetadata(h)));
})
.then(metadatas => {
@@ -562,57 +819,24 @@ define([
};
ExecutePipeline.prototype.createPointers = function (node, files, cb) {
var pointers = this.core.getPointerNames(node).filter(name => name !== 'base'),
nIds = pointers.map(p => this.core.getPointerPath(node, p));
var pointers,
nIds;
this.logger.info('Creating pointers file...');
pointers = this.core.getPointerNames(node)
.filter(name => name !== 'base')
.filter(id => this.core.getPointerPath(node, id) !== null);
nIds = pointers.map(p => this.core.getPointerPath(node, p));
files.ptrAssets = {};
Q.all(
nIds.map(nId => this.core.loadByPath(this.rootNode, nId))
nIds.map(nId => this.getPtrCodeHash(nId))
)
.then(nodes => {
var executePlugin = (pluginId, config, callback) => {
// Call the Interpreter manager in a Q.ninvoke friendly way
// I need to create a custom context for the given plugin:
// - Set the activeNode to the given referenced node
// - If the activeNode is namespaced, set META to the given namespace
//
// FIXME: Check if it is running in the browser or on the server
WebGMEGlobal.Client.runBrowserPlugin(pluginId, config, (err, result) => {
if (!result.success) {
return callback(result.getError());
}
this.logger.info('Finished calling ' + pluginId);
callback(null, result.artifacts);
});
};
return Q.all(
nodes.map(ptrNode => {
// Look up the plugin to use
var metanode = this.core.getMetaType(ptrNode),
pluginId;
pluginId = this.core.getRegistry(ptrNode, 'validPlugins').split(' ').shift();
this.logger.info(`generating code for ${this.core.getAttribute(ptrNode, 'name')} using ${pluginId}`);
var context = WebGMEGlobal.Client.getCurrentPluginContext(pluginId);
context.managerConfig.namespace = this.core.getNamespace(metanode);
context.managerConfig.activeNode = this.core.getPath(ptrNode);
// Load and run the plugin
return Q.nfcall(executePlugin, pluginId, context);
})
);
})
.then(resultHashes => {
var name = this.core.getAttribute(node, 'name');
this.logger.info(`Pointer generation for ${name} FINISHED!`);
resultHashes.forEach((hashes, index) => {
// Grab the first asset for now
// FIXME
files.ptrAssets[`pointers/${pointers[index]}/init.lua`] = hashes[0];
resultHashes.forEach((hash, index) => {
files.ptrAssets[`pointers/${pointers[index]}/init.lua`] = hash;
});
return cb(null, files);
})
@@ -625,14 +849,23 @@ define([
ExecutePipeline.prototype.createOutputs = function (node, files) {
// For each of the output types, grab their serialization functions and
// create the `outputs/init.lua` file
this.logger.info('Creating outputs/init.lua...');
return this.getOutputs(node)
.then(outputs => {
var outputTypes = outputs
// Get the serialize functions for each
.map(tuple => [tuple[1], this.core.getAttribute(tuple[2], 'serialize')]);
.map(tuple => {
var node = tuple[2],
serFn = this.core.getAttribute(node, 'serialize');
// Remove duplicates
// TODO
if (this.isMetaTypeOf(node, this.META.Complex)) {
// Complex objects are expected to define their own
// serialize methods
serFn = 'if data ~= nil then data:serialize(path) end';
}
return [tuple[1], serFn];
});
files['outputs/init.lua'] = _.template(Templates.SERIALIZE)({types: outputTypes});
});
@@ -665,16 +898,12 @@ define([
};
ExecutePipeline.prototype.createEntryFile = function (node, files) {
this.logger.info('Creating entry files...');
return this.getOutputs(node)
.then(outputs => {
var name = this.core.getAttribute(node, 'name'),
content = {};
// sort the outputs by the return values?
if (outputs.length > 1) {
this.logger.error('Multiple outputs not yet supported!');
}
// inputs and outputs
content.name = name;
content.outputs = outputs;
@@ -684,6 +913,7 @@ define([
};
ExecutePipeline.prototype.createMainFile = function (node, files) {
this.logger.info('Creating main file...');
return this.getInputs(node)
.then(inputs => {
var name = this.core.getAttribute(node, 'name'),
@@ -694,10 +924,12 @@ define([
};
// Get input data arguments
content.inputs = inputs;
content.inputs = inputs
.map(pair => [pair[0], !this.core.getAttribute(pair[2], 'data')]); // remove empty inputs
// Defined variables for each pointers
content.pointers = pointers;
content.pointers = pointers
.map(id => [id, this.core.getPointerPath(node, id) === null]);
// Add remaining code
content.code = code;
@@ -707,12 +939,20 @@ define([
};
ExecutePipeline.prototype.createAttributeFile = function (node, files) {
var skip = ['outputs', 'inputs'],
var skip = ['code'],
numRegex = /^\d+\.?\d*((e|e-)\d+)?$/,
table;
this.logger.info('Creating attributes file...');
table = '{\n\t' + this.core.getAttributeNames(node)
.filter(attr => skip.indexOf(attr) === -1)
.map(name => [name, JSON.stringify(this.core.getAttribute(node, name))])
.map(name => {
var value = this.core.getAttribute(node, name);
if (!numRegex.test(value)) {
value = `"${value}"`;
}
return [name, value];
})
.map(pair => pair.join(' = '))
.join(',\n\t') + '\n}';
@@ -720,21 +960,6 @@ define([
};
//////////////////////////// Special Operations ////////////////////////////
ExecutePipeline.prototype.getLocalOpType = function (node) {
var type;
for (var i = LocalExecutor.TYPES.length; i--;) {
type = LocalExecutor.TYPES[i];
if (!this.META[type]) {
this.logger.warn(`Missing local operation: ${type}`);
continue;
}
if (this.isMetaTypeOf(node, this.META[type])) {
return type;
}
}
return null;
};
ExecutePipeline.prototype.executeLocalOperation = function (type, node) {
// Retrieve the given LOCAL_OP type
if (!this[type]) {
@@ -745,7 +970,11 @@ define([
return this[type](node);
};
_.extend(ExecutePipeline.prototype, LocalExecutor.prototype);
_.extend(
ExecutePipeline.prototype,
LocalExecutor.prototype,
PtrCodeGen.prototype
);
return ExecutePipeline;
});
+18 -10
Ver Arquivo
@@ -1,23 +1,31 @@
{
"id": "ExecutePipeline",
"name": "ExecutePipeline",
"name": "Execute Pipeline",
"version": "0.1.0",
"description": "",
"icon": {
"class": "glyphicon glyphicon-cog",
"class": "glyphicon glyphicon-random",
"src": ""
},
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"configStructure": [
{
"name": "debug",
"displayName": "Debug Mode",
"description": "Download all files for each operation",
"value": false,
"valueType": "boolean",
"readOnly": false
}
{
"name": "snapshot",
"displayName": "Snapshot",
"description": "Freeze the operation definitions and attributes at current value",
"value": true,
"valueType": "boolean",
"readOnly": false
},
{
"name": "debug",
"displayName": "Debug Mode",
"description": "Download all files for each operation",
"value": false,
"valueType": "boolean",
"readOnly": false
}
]
}
@@ -1,4 +1,6 @@
require 'paths'
local path = 'inputs/<%= name %>/<%= filename %>'
local abs_path = paths.concat('inputs', '<%= name %>', '<%= filename %>')
<%= code %>
+2 -2
Ver Arquivo
@@ -1,7 +1,7 @@
-- run the <%= name %> and serialize the results
print('\n############### Running "<%= name %>" Operation ############### ')
print('\n############### Running "<%= name.replace(/'/g, '\\\'') %>" Operation ############### ')
results = require './main'
print('############### "<%= name %>" Operation Complete! ###############')
print('############### "<%= name.replace(/'/g, '\\\'') %>" Operation Complete! ###############')
-- serialize by type
outputs = require './outputs'
@@ -11,7 +11,9 @@ define([
DESERIALIZE
) {
var BASH = 'th init.lua 2>&1';
return {
BASH,
ENTRY,
MAIN,
SERIALIZE,
+10 -7
Ver Arquivo
@@ -1,11 +1,14 @@
-- input data
<% inputs.forEach(function(pair) { var input = pair[0]%>
<%= input %> = require './inputs/<%= input %>'
<% }); %>
-- load custom layers
require './custom-layers'
-- load references
<% pointers.forEach(function(pointer) { %><%= pointer %> = require './pointers/<%= pointer %>'
<% }); %>
-- load custom class definitions
require './classes'
-- input data<% inputs.forEach(function(pair) { var input = pair[0], isNil = pair[1];%>
<%= isNil ? 'local ' : ''%><%= input %> = <% if (isNil) { %>nil<% } else { %>require './inputs/<%= input %>'<%}}); %>
-- load references<% pointers.forEach(function(pair) { var pointer = pair[0], isNil = pair[1];%>
<%= isNil ? 'local ' : ''%><%= pointer %> = <% if (isNil) { %>nil<% } else { %>require './pointers/<%= pointer %>'<%}}); %>
attributes = require './attributes'
-- main operation code for <%= name %>
@@ -29,6 +29,7 @@ define([
* @classdesc This class represents the plugin GenerateArchitecture.
* @constructor
*/
var INDEX = '__index__';
var GenerateArchitecture = function () {
// Call base class' constructor.
PluginBase.call(this);
@@ -42,57 +43,166 @@ define([
GenerateArchitecture.prototype.constructor = GenerateArchitecture;
GenerateArchitecture.prototype.main = function () {
this.addCustomLayersToMeta();
this.LayerDict = createLayerDict(this.core, this.META);
this.uniqueId = 2;
this._oldTemplateSettings = _.templateSettings;
return PluginBase.prototype.main.apply(this, arguments);
};
GenerateArchitecture.prototype.addCustomLayersToMeta = function () {
var metaDict = this.core.getAllMetaNodes(this.rootNode);
Object.keys(metaDict).map(id => metaDict[id])
// Get all custom layers
.filter(node => this.core.isTypeOf(node, this.META.Layer))
// Add them to the meta
.forEach(node => this.META[this.core.getAttribute(node, 'name')] = node);
};
GenerateArchitecture.prototype.createOutputFiles = function (tree) {
var layers = tree[Constants.CHILDREN],
//initialLayers,
result = {},
template,
snippet,
code,
args;
code = 'require \'nn\'\n';
code = [
'require \'nn\'',
'',
'local net = nn.Sequential()'
].join('\n');
//initialLayers = layers.filter(layer => layer[Constants.PREV].length === 0);
// Add an index to each layer
layers.forEach((l, index) => l[INDEX] = index);
// Start with sequential (just one input)
for (var i = 0; i < layers.length; i++) {
if (layers[i][Constants.NEXT].length > 1) {
// no support for
this.logger.error('No support for parallel layers... yet');
break;
} else {
// args
args = this.createArgString(layers[i]);
template = _.template('net:add(nn.{{= name }}' + args + ')');
snippet = template(layers[i]);
code += '\n' + snippet;
}
// Define custom layers
if (this.getCurrentConfig().standalone) {
code += this.genLayerDefinitions(layers);
}
code += '\n\nreturn net';
code += this.genArchCode(layers);
result[tree.name + '.lua'] = code;
_.templateSettings = this._oldTemplateSettings; // FIXME: Fix this in SimpleNodes
return result;
};
GenerateArchitecture.prototype.createArgString = function (layer) {
return '(' + this.LayerDict[layer.name].map(arg => {
var value = layer[arg.name];
// Infer if value is unset and infer.dimensionality is set
if (!value && arg.infer === 'dimensionality') {
value = dimensionality(layer[Constants.PREV][0]);
}
return value;
}).join(', ') + ')';
GenerateArchitecture.prototype.genArchCode = function (layers) {
return [
this.createSequential(layers[0], 'net').code,
'\nreturn net'
].join('\n');
};
GenerateArchitecture.prototype.createSequential = function (layer, name) {
var next = layer[Constants.NEXT][0],
args,
template,
snippet,
snippets,
code = `\nlocal ${name} = nn.Sequential()`,
group,
i,
result;
while (layer) {
// if there is only one successor, just add the given layer
if (layer[Constants.PREV].length > 1) { // sequential layers are over
next = layer; // the given layer will be added by the caller
break;
} else { // add the given layer
args = this.createArgString(layer);
template = _.template(name + ':add(nn.{{= name }}' + args + ')');
snippet = template(layer);
code += '\n' + snippet;
}
while (layer && layer[Constants.NEXT].length > 1) { // concat/parallel
// if there is a fork, recurse and add a concat layer
this.logger.debug(`detected fork of size ${layer[Constants.NEXT].length}`);
snippets = layer[Constants.NEXT].map(nlayer =>
this.createSequential(nlayer, 'net_'+(this.uniqueId++)));
code += '\n' + snippets.map(snippet => snippet.code).join('\n');
// Make sure all snippets end at the same concat node
// Until all snippets end at the same concat node
snippets.sort((a, b) => a.endIndex < b.endIndex ? -1 : 1);
group = [];
while (snippets.length > 0) {
// Add snippets to the group
i = 0;
while (i < snippets.length &&
snippets[0].endIndex === snippets[i].endIndex) {
group.push(snippets[i]);
i++;
}
// Add concat layer
layer = group[0].next;
if (layer) {
args = this.createArgString(layer);
code += `\n\nlocal concat_${layer[INDEX]} = nn.Concat${args}\n` +
group.map(snippet =>
`concat_${layer[INDEX]}:add(${snippet.name})`)
.join('\n') + `\n\n${name}:add(concat_${layer[INDEX]})`;
next = layer[Constants.NEXT][0];
} else {
next = null; // no next layers
}
// Remove the updated snippets
this.logger.debug('removing ' + i + ' snippet(s)');
snippets.splice(0, i);
// merge the elements in the group
if (snippets.length) { // prepare next iteration
result = this.createSequential(next, 'net_'+(this.uniqueId++));
code += result.code;
group = [result];
this.logger.debug('updating group ('+ snippets.length+ ' left)');
}
}
}
layer = next;
next = layer && layer[Constants.NEXT][0];
}
return {
code: code,
name: name,
endIndex: next ? next[INDEX] : Infinity,
next: next
};
};
GenerateArchitecture.prototype.createArgString = function (layer) {
return '(' + this.LayerDict[layer.name]
.map(arg => layer[arg.name])
.filter(GenerateArchitecture.isSet)
.join(', ') + ')';
};
GenerateArchitecture.isSet = function (value) {
return !(value === undefined || value === null || value === '');
};
GenerateArchitecture.prototype.genLayerDefinitions = function(layers) {
var code = '',
customLayerId = this.core.getPath(this.META.CustomLayer),
customLayers = layers.filter(layer => { // Get the custom layers
var node = this.META[layer.name];
return this.core.getMixinPaths(node).indexOf(customLayerId) !== -1;
});
if (customLayers.length) {
code += '\n-------------- Custom Layer Definitions --------------\n\n';
code += customLayers.map(layer => layer.code).join('\n');
code += '\n\n-------------- Network --------------\n';
}
return code;
};
return GenerateArchitecture;
});
+12 -3
Ver Arquivo
@@ -1,13 +1,22 @@
{
"id": "GenerateArchitecture",
"name": "Generate Architecture",
"name": "Generate Torch Code",
"version": "0.1.0",
"description": "Generate torch architecture code",
"icon": {
"src": "",
"class": "glyphicon glyphicon-ok-circle"
"class": "glyphicon glyphicon-file"
},
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"configStructure": []
"configStructure": [
{
"name": "standalone",
"displayName": "Standalone",
"description": "Prepend custom layer definitions",
"value": false,
"valueType": "boolean",
"readOnly": false
}
]
}
@@ -0,0 +1,69 @@
/*globals define*/
/*jshint node:true, browser:true*/
define([
'text!./metadata.json',
'plugin/PluginBase'
], function (
pluginMetadata,
PluginBase
) {
'use strict';
pluginMetadata = JSON.parse(pluginMetadata);
/**
* Initializes a new instance of GenerateCriterion.
* @class
* @augments {PluginBase}
* @classdesc This class represents the plugin GenerateCriterion.
* @constructor
*/
var GenerateCriterion = function () {
// Call base class' constructor.
PluginBase.call(this);
this.pluginMetadata = pluginMetadata;
};
/**
* Metadata associated with the plugin. Contains id, name, version, description, icon, configStructue etc.
* This is also available at the instance at this.pluginMetadata.
* @type {object}
*/
GenerateCriterion.metadata = pluginMetadata;
// Prototypical inheritance from PluginBase.
GenerateCriterion.prototype = Object.create(PluginBase.prototype);
GenerateCriterion.prototype.constructor = GenerateCriterion;
/**
* Main function for the plugin to execute. This will perform the execution.
* Notes:
* - Always log with the provided logger.[error,warning,info,debug].
* - Do NOT put any user interaction logic UI, etc. inside this method.
* - callback always has to be called even if error happened.
*
* @param {function(string, plugin.PluginResult)} callback - the result callback
*/
GenerateCriterion.prototype.main = function (callback) {
// Generate the code for the criterion layer and return a file
var name = this.core.getAttribute(this.activeNode, 'name'),
code = `require 'nn'\nreturn nn.${name}()`,
filename = `${name}.lua`;
// Using the logger.
this.logger.debug(`Generating code for ${name} criterion layer.`);
// Save the file
this.blobClient.putFile(filename, code)
.then(hash => {
this.result.setSuccess(true);
this.result.addArtifact(hash);
callback(null, this.result);
})
.catch(err => callback(err, this.result));
};
return GenerateCriterion;
});
+14
Ver Arquivo
@@ -0,0 +1,14 @@
{
"id": "GenerateCriterion",
"name": "Generate Criterion Code",
"version": "0.1.0",
"description": "",
"icon": {
"class": "glyphicon glyphicon-cog",
"src": ""
},
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"writeAccessRequired": false,
"configStructure": []
}
@@ -0,0 +1,344 @@
/*globals define, _*/
/*jshint node:true, browser:true*/
/**
* Generated by PluginGenerator 1.7.0 from webgme on Sat Jun 04 2016 18:01:54 GMT-0500 (CDT).
* A plugin that inherits from the PluginBase. To see source code documentation about available
* properties and methods visit %host%/docs/source/PluginBase.html.
*/
define([
'text!./metadata.json',
'plugin/PluginBase',
'deepforge/plugin/PtrCodeGen',
'q'
], function (
pluginMetadata,
PluginBase,
PtrCodeGen,
Q
) {
'use strict';
pluginMetadata = JSON.parse(pluginMetadata);
var HEADER_LENGTH = 60;
/**
* Initializes a new instance of GenerateExecFile.
* @class
* @augments {PluginBase}
* @classdesc This class represents the plugin GenerateExecFile.
* @constructor
*/
var GenerateExecFile = function () {
// Call base class' constructor.
PluginBase.call(this);
this.pluginMetadata = pluginMetadata;
this._srcIdFor = {}; // input path -> output data node path
this._nameFor = {}; // input path -> opname
this._dataNameFor = {};
this._opNames = {};
// topo sort stuff
this._nextOps = {};
this._incomingCnts = {};
this._operations = {};
this.activeNodeId = null;
this.activeNodeDepth = null;
};
/**
* Metadata associated with the plugin. Contains id, name, version, description, icon, configStructue etc.
* This is also available at the instance at this.pluginMetadata.
* @type {object}
*/
GenerateExecFile.metadata = pluginMetadata;
// Prototypical inheritance from PluginBase.
GenerateExecFile.prototype = Object.create(PluginBase.prototype);
GenerateExecFile.prototype.constructor = GenerateExecFile;
/**
* Main function for the plugin to execute. This will perform the execution.
* Notes:
* - Always log with the provided logger.[error,warning,info,debug].
* - Do NOT put any user interaction logic UI, etc. inside this method.
* - callback always has to be called even if error happened.
*
* @param {function(string, plugin.PluginResult)} callback - the result callback
*/
GenerateExecFile.prototype.main = function (callback) {
// Get all the children and call generate exec file
this.activeNodeId = this.core.getPath(this.activeNode);
this.activeNodeDepth = this.activeNodeId.split('/').length + 1;
if (this.isMetaTypeOf(this.activeNode, this.META.Execution)) {
this.activeNodeDepth++;
}
return this.core.loadChildren(this.activeNode)
.then(nodes => this.createExecFile(nodes))
.then(code => this.blobClient.putFile('init.lua', code))
.then(hash => {
this.result.addArtifact(hash);
this.result.setSuccess(true);
callback(null, this.result);
})
.fail(err => callback(err));
};
GenerateExecFile.prototype.createExecFile = function (children) {
// Convert opNodes' jobs to the nested operations
var opNodes,
nodes;
return this.unpackJobs(children)
.then(_nodes => {
nodes = _nodes;
opNodes = nodes
.filter(node => this.isMetaTypeOf(node, this.META.Operation));
return Q.all(nodes.map(node => this.registerNameAndData(node)));
})
.then(() => Q.all(opNodes.map(node => this.createOperation(node))))
.then(operations => {
var nextIds = opNodes.map(n => this.core.getPath(n))
.filter(id => !this._incomingCnts[id]);
operations.forEach(op => this._operations[op.id] = op);
// Toposort and concat!
return this.combineOpNodes(nextIds);
})
.fail(err => this.logger.error(err));
};
GenerateExecFile.prototype.unpackJobs = function (nodes) {
return Q.all(
nodes.map(node => {
if (!this.isMetaTypeOf(node, this.META.Job)) {
return node;
}
return this.core.loadChildren(node)
.then(children =>
children.find(c => this.isMetaTypeOf(c, this.META.Operation))
);
})
);
};
GenerateExecFile.prototype.combineOpNodes = function (opIds) {
var nextIds = [],
dstIds,
code,
id;
// Combine all nodes with incoming cnts of 0
code = opIds.map(id => this._operations[id].code).join('\n');
// Decrement all next ops
dstIds = opIds.map(id => this._nextOps[id])
.reduce((l1, l2) => l1.concat(l2), []);
for (var i = dstIds.length; i--;) {
id = dstIds[i];
if (--this._incomingCnts[id] === 0) {
nextIds.push(id);
}
}
// append
return [
code,
nextIds.length ? this.combineOpNodes(nextIds) : ''
].join('\n');
};
GenerateExecFile.prototype.registerNameAndData = function (node) {
var name = this.core.getAttribute(node, 'name'),
id = this.core.getPath(node),
basename = name,
i = 2;
if (this.isMetaTypeOf(node, this.META.Operation)) {
// Get a unique operation name
while (this._opNames[name]) {
name = basename + '_' + i;
i++;
}
// register the unique name
this._opNames[name] = true;
this._nameFor[id] = name;
// For operations, register all output data node names by path
return this.core.loadChildren(node)
.then(cntrs => {
var cntr = cntrs.find(n => this.isMetaTypeOf(n, this.META.Outputs));
return this.core.loadChildren(cntr);
})
.then(outputs => {
outputs.forEach(output => {
var dataId = this.core.getPath(output);
name = this.core.getAttribute(output, 'name');
this._dataNameFor[dataId] = name;
});
});
// For each input data node, register the associated output id
} else if (this.isMetaTypeOf(node, this.META.Transporter)) {
var outputData = this.core.getPointerPath(node, 'src'),
inputData = this.core.getPointerPath(node, 'dst'),
srcOpId = this.getOpIdFor(outputData),
dstOpId = this.getOpIdFor(inputData);
this._srcIdFor[inputData] = outputData;
// Store the next operation ids for the op id
if (!this._nextOps[srcOpId]) {
this._nextOps[srcOpId] = [];
}
this._nextOps[srcOpId].push(dstOpId);
// Increment the incoming counts for each dst op
this._incomingCnts[dstOpId] = this._incomingCnts[dstOpId] || 0;
this._incomingCnts[dstOpId]++;
}
};
GenerateExecFile.prototype.getOpIdFor = function (dataId) {
var ids = dataId.split('/'),
depth = ids.length;
ids.splice(this.activeNodeDepth - depth);
return ids.join('/');
};
// For each operation...
// - unpack the inputs from prev ops
// - add the attributes table (if used)
// - check for '\<attributes\>' in code
// - add the references
// - generate the code
// - replace the `return <thing>` w/ `<ref-name> = <thing>`
GenerateExecFile.prototype.createOperation = function (node) {
var id = this.core.getPath(node),
operation = {};
operation.name = this._nameFor[id];
operation.id = id;
operation.code = this.core.getAttribute(node, 'code');
// Update the 'code' attribute
// Change the last return statement to assign the results to a table
operation.code = this.assignResultToVar(operation.code,
`${operation.name}_results`);
// Get all the input names (and sources)
return this.core.loadChildren(node)
.then(containers => {
var inputs;
inputs = containers
.find(cntr => this.isMetaTypeOf(cntr, this.META.Inputs));
this.logger.info(`${name} has ${containers.length} cntrs`);
return this.core.loadChildren(inputs);
})
.then(data => {
// Get the input names and sources
var inputNames = data.map(d => this.core.getAttribute(d, 'name')),
ids = data.map(d => this.core.getPath(d)),
srcIds = ids.map(id => this._srcIdFor[id]);
operation.inputs = inputNames.map((name, i) => {
var id = srcIds[i],
srcDataName = this._dataNameFor[id],
srcOpId = this.getOpIdFor(id),
srcOpName = this._nameFor[srcOpId];
return `local ${name} = ${srcOpName}_results.${srcDataName}`;
});
return operation;
})
.then(operation => {
// For each reference, run the plugin and retrieve the generated code
operation.refNames = this.core.getPointerNames(node)
.filter(name => name !== 'base');
var refs = operation.refNames
.map(ref => [ref, this.core.getPointerPath(node, ref)]);
return Q.all(
refs.map(pair => this.genPtrSnippet.apply(this, pair))
);
})
.then(codeFiles => {
operation.refs = codeFiles;
this.genOperationCode(operation);
return operation;
});
};
GenerateExecFile.prototype.genPtrSnippet = function (ptrName, pId) {
return this.getPtrCodeHash(pId)
.then(hash => this.blobClient.getObjectAsString(hash))
.then(code => this.createHeader(`creating ${ptrName}`, 40) + '\n' +
this.assignResultToVar(code, ptrName));
};
GenerateExecFile.prototype.createHeader = function (title, length) {
var len;
title = ` ${title} `;
length = length || HEADER_LENGTH;
len = Math.max(
Math.floor((length - title.length)/2),
2
);
return [
'',
title,
''
].join(new Array(len+1).join('-')) + '\n';
};
GenerateExecFile.prototype.genOperationCode = function (operation) {
var header = this.createHeader(`"${operation.name}" Operation`),
codeParts = [];
codeParts.push(header);
if (operation.inputs.length) {
codeParts.push(operation.inputs.join('\n'));
}
if (operation.refs.length) {
codeParts.push(operation.refs.join('\n'));
}
codeParts.push(operation.code);
codeParts.push('');
operation.code = codeParts.join('\n');
return operation;
};
GenerateExecFile.prototype.assignResultToVar = function (code, name) {
var i = code.lastIndexOf('return');
return code.substring(0, i) +
code.substring(i)
.replace('return', `local ${name} = `);
};
_.extend(GenerateExecFile.prototype, PtrCodeGen.prototype);
return GenerateExecFile;
});
+14
Ver Arquivo
@@ -0,0 +1,14 @@
{
"id": "GenerateExecFile",
"name": "Generate Execution File",
"version": "0.1.0",
"description": "",
"icon": {
"class": "glyphicon glyphicon-cog",
"src": ""
},
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"writeAccessRequired": false,
"configStructure": []
}
+128
Ver Arquivo
@@ -0,0 +1,128 @@
/*globals define*/
/*jshint node:true, browser:true*/
/**
* Generated by PluginGenerator 1.7.0 from webgme on Tue Jun 07 2016 11:25:09 GMT-0500 (CDT).
* A plugin that inherits from the PluginBase. To see source code documentation about available
* properties and methods visit %host%/docs/source/PluginBase.html.
*/
define([
'text!./metadata.json',
'plugin/PluginBase',
'q'
], function (
pluginMetadata,
PluginBase,
Q
) {
'use strict';
pluginMetadata = JSON.parse(pluginMetadata);
/**
* Initializes a new instance of ImportArtifact.
* @class
* @augments {PluginBase}
* @classdesc This class represents the plugin ImportArtifact.
* @constructor
*/
var ImportArtifact = function () {
// Call base class' constructor.
PluginBase.call(this);
this.pluginMetadata = pluginMetadata;
};
/**
* Metadata associated with the plugin. Contains id, name, version, description, icon, configStructue etc.
* This is also available at the instance at this.pluginMetadata.
* @type {object}
*/
ImportArtifact.metadata = pluginMetadata;
// Prototypical inheritance from PluginBase.
ImportArtifact.prototype = Object.create(PluginBase.prototype);
ImportArtifact.prototype.constructor = ImportArtifact;
/**
* Main function for the plugin to execute. This will perform the execution.
* Notes:
* - Always log with the provided logger.[error,warning,info,debug].
* - Do NOT put any user interaction logic UI, etc. inside this method.
* - callback always has to be called even if error happened.
*
* @param {function(string, plugin.PluginResult)} callback - the result callback
*/
ImportArtifact.prototype.main = function (callback) {
var self = this,
config = this.getCurrentConfig(),
hash = config.dataHash,
baseName = config.dataTypeId,
name,
baseType,
dataNode,
metaDict,
metanodes;
// Create node of type "typeId" in the activeNode and set the hash, name
metaDict = this.core.getAllMetaNodes(this.activeNode);
metanodes = Object.keys(metaDict).map(id => metaDict[id]);
baseType = metanodes.find(node =>
this.core.getAttribute(node, 'name') === baseName
);
if (!baseType) {
callback(`Could not find data type "${baseName}"`, this.result);
return;
}
// Get the base node
this.getArtifactsDir()
.then(targetDir => {
dataNode = this.core.createNode({
base: baseType,
parent: targetDir
});
this.core.setAttribute(dataNode, 'data', hash);
baseName = this.core.getAttribute(baseType, 'name');
var getName;
if (config.name) {
getName = Q().then(() => config.name);
} else {
getName = this.blobClient.getMetadata(hash)
.then(md => {
name = baseName[0].toLowerCase() + baseName.substring(1);
if (md) {
name = md.name.replace(/\.[^\.]*?$/, '');
}
return name;
});
}
return getName;
})
.then(name => this.core.setAttribute(dataNode, 'name', name))
.then(() => this.save(`Uploaded "${name}" data`))
.then(function () {
self.result.setSuccess(true);
callback(null, self.result);
})
.fail(function (err) {
callback(err, self.result);
});
};
ImportArtifact.prototype.getArtifactsDir = function() {
// Find the artifacts dir
return this.core.loadChildren(this.rootNode)
.then(children => children
.find(child => this.core.getAttribute(child, 'name') === 'MyArtifacts') ||
this.activeNode
);
};
return ImportArtifact;
});
+31
Ver Arquivo
@@ -0,0 +1,31 @@
{
"id": "ImportArtifact",
"name": "Import Artifact",
"version": "0.1.0",
"description": "",
"icon": {
"class": "glyphicon glyphicon-cloud-upload",
"src": ""
},
"disableServerSideExecution": false,
"disableBrowserSideExecution": false,
"writeAccessRequired": true,
"configStructure": [
{
"name": "name",
"displayName": "Data name",
"description": "Optional name for artifact",
"value": "",
"valueType": "string",
"readOnly": false
},
{
"name": "dataHash",
"displayName": "Data to upload",
"description": "",
"value": "",
"valueType": "asset",
"readOnly": false
}
]
}
+15 -7
Ver Arquivo
@@ -57,12 +57,17 @@ define([
this.blobClient.getMetadata(srcHash)
.then(mdata => { // Create the new model
var name = mdata.name.replace('.lua', '');
this.tgtNode = this.core.createNode({
base: this.META.Architecture,
parent: this.activeNode
});
this.core.setAttribute(this.tgtNode, 'name', name);
// If the current node is an architecture, assume we are just extending it
this.importedName = mdata.name.replace('.lua', '');
if (this.isMetaTypeOf(this.activeNode, this.META.Architecture)) {
this.tgtNode = this.activeNode;
} else { // Create a new architecture
this.tgtNode = this.core.createNode({
base: this.META.Architecture,
parent: this.activeNode
});
this.core.setAttribute(this.tgtNode, 'name', this.importedName);
}
return this.blobClient.getObjectAsString(srcHash);
})
.then(src => { // Retrieved the source code
@@ -81,7 +86,10 @@ define([
return this.save('ImportTorch updated model.');
})
.then(() => { // changes saved successfully
var name = this.importedName;
this.result.setSuccess(true);
this.createMessage(this.tgtNode,
`Successfully imported ${name} architecture`);
callback(null, this.result);
})
.fail(err =>
@@ -93,7 +101,7 @@ define([
ImportTorch.prototype.loadNNMock = function () {
// This needs a refactor...
// createNN(this)
var lib = createNNSearcher(this).bind(this.context);
var lib = createNNSearcher(this, this.context).bind(this.context);
// Create a "searcher" to allow this 'nn' to be in the lib path
this.context._G.get('package').set('searchers', [function(name) {
+24 -2
Ver Arquivo
@@ -11,12 +11,23 @@ define([
) {
'use strict';
var createSearcher = function(plugin) {
var createSearcher = function(plugin, context) {
var core = plugin.core,
META = plugin.META,
logger = plugin.logger.fork('nn'),
parent = plugin.tgtNode,
LayerDict = createLayerDict(core, META);
LayerDict = createLayerDict(core, META),
helpers = context.__helpers,
oldSet = helpers.__set,
isSetting = false;
// Override the helper's '__set' method to detect
// if the code is in the middle of a "set".
helpers.__set = function() {
isSetting = true;
oldSet.apply(this, arguments);
isSetting = false;
};
var connect = function(src, dst) {
var conn = core.createNode({
@@ -145,6 +156,7 @@ define([
var CreateLayer = function(type) {
var res = luajs.newContext()._G,
attrs = [].slice.call(arguments, 1),
ltGet = luajs.types.LuaTable.prototype.get,
node;
if (LAYERS[type]) {
@@ -165,6 +177,16 @@ define([
}
}
}
// Override get
res.get = function noNilGet(value) {
var result = ltGet.call(this, value);
if (!result && !isSetting) {
throw Error(`"${value}" is not supported for ${type}`);
}
return result;
};
return res;
};
Arquivo binário não exibido.
-3
Ver Arquivo
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:b3c5a08313ef9d966fde9893f89ea7b4a6f8d00f6629e1234fdca71911acfe6a
size 455762
-3
Ver Arquivo
@@ -1,3 +0,0 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0f7228f775590d69e1215d25afc299809755a6e820fbd5cfdcd1e9bb3abdaf3a
size 23753
Arquivo binário não exibido.
Arquivo binário não exibido.
Arquivo binário não exibido.
Arquivo binário não exibido.
Arquivo binário não exibido.
Arquivo binário não exibido.
+27 -39
Ver Arquivo
@@ -5,24 +5,12 @@
"panel": "panels/AutoViz/AutoVizPanel",
"DEBUG_ONLY": false
},
{
"id": "EasyDAG",
"title": "EasyDAG",
"panel": "panels/EasyDAG/EasyDAGPanel",
"DEBUG_ONLY": false
},
{
"id": "ArchEditor",
"title": "ArchEditor",
"panel": "panels/ArchEditor/ArchEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "TextEditor",
"title": "TextEditor",
"panel": "panels/TextEditor/TextEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "OperationEditor",
"title": "OperationEditor",
@@ -43,26 +31,8 @@
},
{
"id": "RootViz",
"title": "RootViz",
"panel": "panels/RootViz/RootVizPanel",
"DEBUG_ONLY": false
},
{
"id": "TilingViz",
"title": "TilingViz",
"panel": "panels/TilingViz/TilingVizPanel",
"DEBUG_ONLY": false
},
{
"id": "OperationCodeEditor",
"title": "OperationCodeEditor",
"panel": "panels/OperationCodeEditor/OperationCodeEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "OperationInterfaceEditor",
"title": "OperationInterfaceEditor",
"panel": "panels/OperationInterfaceEditor/OperationInterfaceEditorPanel",
"title": "MainView",
"panel": "panels/MainView/MainViewPanel",
"DEBUG_ONLY": false
},
{
@@ -72,15 +42,33 @@
"DEBUG_ONLY": false
},
{
"id": "SerializeEditor",
"title": "SerializeEditor",
"panel": "panels/SerializeEditor/SerializeEditorPanel",
"id": "LogViewer",
"title": "LogViewer",
"panel": "panels/LogViewer/LogViewerPanel",
"DEBUG_ONLY": false
},
{
"id": "DeserializeEditor",
"title": "DeserializeEditor",
"panel": "panels/DeserializeEditor/DeserializeEditorPanel",
"id": "LayerEditor",
"title": "LayerEditor",
"panel": "panels/LayerEditor/LayerEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "ClassCodeEditor",
"title": "ClassCodeEditor",
"panel": "panels/ClassCodeEditor/ClassCodeEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "ClassEditor",
"title": "ClassEditor",
"panel": "panels/ClassEditor/ClassEditorPanel",
"DEBUG_ONLY": false
},
{
"id": "PipelineIndex",
"title": "PipelineIndex",
"panel": "panels/PipelineIndex/PipelineIndexPanel",
"DEBUG_ONLY": false
}
]
]
@@ -1,15 +1,14 @@
/*globals define */
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Tue May 17 2016 11:25:46 GMT-0400 (EDT).
*/
define([
'deepforge/globals',
'panels/EasyDAG/EasyDAGControl',
'js/NodePropertyNames',
'js/Utils/ComponentSettings',
'underscore'
], function (
DeepForge,
EasyDAGControl,
nodePropertyNames,
ComponentSettings,
@@ -23,12 +22,10 @@ define([
DefaultColor: '#ffb74d',
LayerColors: {
Containers: '#ffb74d',
Module: '#ba68c8',
ConvLayer: '#2196f3',
SimpleLayer: '#ff9100',
TransferLayer: '#80deea',
MiscLayers: '#ce93d8',
Criterion: '#7e57c2'
Convolution: '#2196f3',
Simple: '#ff9100',
Transfer: '#80deea',
Misc: '#ce93d8'
}
};
@@ -45,6 +42,16 @@ define([
return 'ArchEditor';
};
ArchEditorControl.prototype.selectedObjectChanged = function(id) {
EasyDAGControl.prototype.selectedObjectChanged.call(this, id);
DeepForge.last.Architecture = id;
if (typeof id === 'string') {
var name = this._client.getNode(id).getAttribute('name');
this._widget.setTitle(name);
}
};
ArchEditorControl.prototype._getObjectDescriptor = function(id) {
var desc = EasyDAGControl.prototype._getObjectDescriptor.call(this, id);
@@ -86,6 +93,7 @@ define([
return desc;
};
////////////////////////// Layer Selection Logic //////////////////////////
ArchEditorControl.prototype._getValidInitialNodes = function() {
return this._client.getChildrenMeta(this._currentNodeId).items
// For now, anything is possible!
@@ -97,7 +105,67 @@ define([
return !this._client.getNode(nodeId).isAbstract();
})
.map(id => this._getObjectDescriptor(id))
.filter(obj => !obj.isConnection && obj.name !== 'Connection');
.filter(obj => !obj.isConnection && obj.name !== 'Connection')
.filter(layer => layer.layerType !== 'Criterion');
};
ArchEditorControl.prototype._getValidSuccessorNodes =
ArchEditorControl.prototype._getValidInitialNodes =
ArchEditorControl.prototype.getNonCriterionLayers = function() {
// Return all (non-criterion) layer types
var metanodes = this._client.getAllMetaNodes(),
layerId,
criterionId,
allLayerIds = [],
layers = [],
i;
for (i = metanodes.length; i--;) {
if (metanodes[i].getAttribute('name') === 'Layer') {
layerId = metanodes[i].getId();
break;
}
}
for (i = metanodes.length; i--;) {
if (layerId) {
if (!metanodes[i].isAbstract() &&
this._client.isTypeOf(metanodes[i].getId(), layerId)) {
if (metanodes[i].getAttribute('name') === 'Criterion') {
criterionId = metanodes[i].getId();
} else {
allLayerIds.push(metanodes[i].getId());
}
}
}
}
// Remove all criterion layers and abstract layers
for (i = allLayerIds.length; i--;) {
if (!this._client.isTypeOf(allLayerIds[i], criterionId)) {
layers.push({node: this._getObjectDescriptor(allLayerIds[i])});
}
}
return layers;
};
ArchEditorControl.prototype._isValidTerminalNode = function() {
return true;
};
// Widget extensions
ArchEditorControl.prototype._initWidgetEventHandlers = function() {
EasyDAGControl.prototype._initWidgetEventHandlers.call(this);
this._widget.getCreateNewDecorator = this.getCreateNewDecorator.bind(this);
};
ArchEditorControl.prototype.getCreateNewDecorator = function() {
return this._client.decoratorManager.getDecoratorForWidget(
'EllipseDecorator',
'EasyDAG'
);
};
return ArchEditorControl;
@@ -5,13 +5,11 @@
*/
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'deepforge/viz/RenameablePanel',
'widgets/ArchEditor/ArchEditorWidget',
'./ArchEditorControl'
], function (
PanelBaseWithHeader,
IActivePanel,
RenameablePanel,
ArchEditorWidget,
ArchEditorControl
) {
@@ -22,11 +20,11 @@ define([
ArchEditorPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'ArchEditorPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
options[RenameablePanel.OPTIONS.LOGGER_INSTANCE_NAME] = 'ArchEditorPanel';
options[RenameablePanel.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
RenameablePanel.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
@@ -37,9 +35,7 @@ define([
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(ArchEditorPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(ArchEditorPanel.prototype, IActivePanel.prototype);
_.extend(ArchEditorPanel.prototype, RenameablePanel.prototype);
ArchEditorPanel.prototype._initialize = function () {
var self = this;
@@ -60,6 +56,7 @@ define([
widget: this.widget
});
this.initializeRenameable();
this.onActivate();
};
@@ -67,7 +64,7 @@ define([
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
ArchEditorPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
RenameablePanel.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
@@ -81,7 +78,7 @@ define([
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
RenameablePanel.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
@@ -0,0 +1,71 @@
/*globals define */
/*jshint browser: true*/
define([
'panels/TextEditor/TextEditorControl',
'underscore',
'text!./DefaultCodeTemplate.ejs'
], function (
TextEditorControl,
_,
CODE_TEMPLATE
) {
'use strict';
var ClassCodeEditorControl,
getBoilerplate = _.template(CODE_TEMPLATE);
ClassCodeEditorControl = function (options) {
options.attributeName = 'code';
TextEditorControl.call(this, options);
};
_.extend(
ClassCodeEditorControl.prototype,
TextEditorControl.prototype
);
// input/output updates are actually activeNode updates
ClassCodeEditorControl.prototype._onUpdate = function (id) {
if (id === this._currentNodeId) {
TextEditorControl.prototype._onUpdate.call(this, id);
}
};
ClassCodeEditorControl.prototype._getObjectDescriptor = function (nodeId) {
var desc = TextEditorControl.prototype._getObjectDescriptor.call(this, nodeId),
node = this._client.getNode(nodeId),
ownCode = node.getOwnAttribute(this.ATTRIBUTE_NAME);
// If the 'text' attribute is not set, and it's not inheriting anything
if (!desc.text && ownCode === undefined) {
desc.text = getBoilerplate(desc);
}
return desc;
};
ClassCodeEditorControl.prototype.saveTextFor = function (id, text) {
// On save, update the node's name
// For now, simply use regex to grab the returned name
var i = text.lastIndexOf('return') + 7,
returned = text.substring(i),
match = returned.match(/[a-zA-Z0-9_]+/),
node = this._client.getNode(id),
nodeName = node.getAttribute('name'),
name;
if (match) {
name = match[0];
}
this._client.startTransaction(`Updating class "${name || nodeName}"`);
if (name) {
this._client.setAttributes(id, 'name', name);
}
TextEditorControl.prototype.saveTextFor.call(this, id, text);
this._client.completeTransaction();
};
return ClassCodeEditorControl;
});
@@ -0,0 +1,101 @@
/*globals define, _, WebGMEGlobal*/
/*jshint browser: true*/
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/ClassCodeEditor/ClassCodeEditorWidget',
'./ClassCodeEditorControl'
], function (
PanelBaseWithHeader,
IActivePanel,
ClassCodeEditorWidget,
ClassCodeEditorControl
) {
'use strict';
var ClassCodeEditorPanel;
ClassCodeEditorPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'ClassCodeEditorPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
//initialize UI
this._initialize();
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(ClassCodeEditorPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(ClassCodeEditorPanel.prototype, IActivePanel.prototype);
ClassCodeEditorPanel.prototype._initialize = function () {
var self = this;
//set Widget title
this.setTitle('');
this.widget = new ClassCodeEditorWidget(this.logger, this.$el);
this.widget.setTitle = function (title) {
self.setTitle(title);
};
this.control = new ClassCodeEditorControl({
logger: this.logger,
client: this._client,
embedded: this._embedded,
widget: this.widget
});
this.onActivate();
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
ClassCodeEditorPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
ClassCodeEditorPanel.prototype.onResize = function (width, height) {
this.logger.debug('onResize --> width: ' + width + ', height: ' + height);
this.widget.onWidgetContainerResize(width, height);
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
ClassCodeEditorPanel.prototype.destroy = function () {
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
ClassCodeEditorPanel.prototype.onActivate = function () {
this.widget.onActivate();
this.control.onActivate();
WebGMEGlobal.KeyboardManager.setListener(this.widget);
WebGMEGlobal.Toolbar.refresh();
};
ClassCodeEditorPanel.prototype.onDeactivate = function () {
this.widget.onDeactivate();
this.control.onDeactivate();
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
return ClassCodeEditorPanel;
});
@@ -0,0 +1,16 @@
torch.class('<%= name %>')
function <%= name %>:__init()
-- TODO: Set constructor behavior
end
function <%= name %>:serialize(filename)
torch.save(filename, self)
end
function <%= name %>.deserialize(filename)
-- TODO: Load (and return) a '<%= name %>' instance saved to the given file
return torch.load(filename)
end
return <%= name %>
@@ -0,0 +1,29 @@
/*globals define */
/*jshint browser: true*/
define([
'panels/TilingViz/TilingVizPanel',
'panels/ClassCodeEditor/ClassCodeEditorPanel',
'underscore'
], function (
TilingViz,
ClassCodeEditor,
_
) {
'use strict';
var ClassEditorPanel;
ClassEditorPanel = function (layoutManager, params) {
TilingViz.call(this, layoutManager, params);
};
//inherit from PanelBaseWithHeader
_.extend(ClassEditorPanel.prototype, TilingViz.prototype);
ClassEditorPanel.prototype.getPanels = function () {
return [ClassCodeEditor];
};
return ClassEditorPanel;
});
@@ -2,9 +2,11 @@
/*jshint browser: true*/
define([
'panels/SerializeEditor/SerializeEditorControl',
'panels/TextEditor/TextEditorControl',
'underscore'
], function (
SerializeEditorControl,
TextEditorControl,
_
) {
@@ -20,15 +22,8 @@ define([
_.extend(
DeserializeEditorControl.prototype,
TextEditorControl.prototype
SerializeEditorControl.prototype
);
// input/output updates are actually activeNode updates
DeserializeEditorControl.prototype._onUpdate = function (id) {
if (id === this._currentNodeId) {
TextEditorControl.prototype._onUpdate.call(this, this._currentNodeId);
}
};
return DeserializeEditorControl;
});
@@ -1,14 +1,16 @@
/*globals define */
/*globals define, WebGMEGlobal */
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Tue May 24 2016 10:15:19 GMT-0500 (CDT).
*/
define([
'js/Constants',
'panels/EasyDAG/EasyDAGControl',
'deepforge/viz/PipelineControl',
'underscore'
], function (
CONSTANTS,
EasyDAGControl,
PipelineControl,
_
@@ -21,6 +23,8 @@ define([
ExecutionViewControl = function (options) {
EasyDAGControl.call(this, options);
this.addedNodes = {};
this.originTerritory = {};
this.originTerritoryId = null;
};
_.extend(
@@ -33,8 +37,62 @@ define([
ExecutionViewControl.prototype.TERRITORY_RULE = {children: 4};
ExecutionViewControl.prototype.DEFAULT_DECORATOR = 'JobDecorator';
ExecutionViewControl.prototype.selectedObjectChanged = function(id) {
EasyDAGControl.prototype.selectedObjectChanged.call(this, id);
if (this._currentNodeId) {
var desc = this.getExecDesc(this._currentNodeId);
this._widget.setExecutionNode(desc);
this.originId = desc.originId;
// Add the originId to the territory and update it!
if (this.originId) {
if (this.originTerritoryId) {
this._client.removeUI(this.originTerritoryId);
this.originTerritory = {};
}
this.originTerritory[this.originId] = {children: 0};
this.originTerritoryId = this._client.addUI(this, events => {
var event = events.find(event => event.eid !== null &&
event.eid === this.originId);
if (!event) { // no relevant events
return;
}
if (event.etype === CONSTANTS.TERRITORY_EVENT_UNLOAD) {
this.originId = null;
this._widget.onOriginDeleted();
} else {
var name = this._client.getNode(this.originId).getAttribute('name');
this._widget.setOriginPipeline(name);
}
});
this._client.updateTerritory(this.originTerritoryId, this.originTerritory);
} else {
this._widget.onOriginDeleted();
if (this.originTerritoryId) {
this._client.removeUI(this.originTerritoryId);
}
}
}
};
ExecutionViewControl.prototype.getExecDesc = function(id) {
var node = this._client.getNode(id);
return {
isSnapshot: node.getAttribute('snapshot'),
createdAt: node.getAttribute('createdAt'),
originId: node.getPointer('origin').to
};
};
ExecutionViewControl.prototype._onLoad = function(id) {
var desc = this._getObjectDescriptor(id);
if (desc.parentId === this._currentNodeId) {
this.addedNodes[id] = true;
EasyDAGControl.prototype._onLoad.call(this, id);
@@ -54,6 +112,11 @@ define([
}
};
// Add the connection detection
ExecutionViewControl.prototype.onOriginClicked = function() {
if (this.originId) {
WebGMEGlobal.State.registerActiveObject(this.originId);
}
};
return ExecutionViewControl;
});
@@ -1,17 +1,17 @@
/*globals define, _, WebGMEGlobal*/
/*globals define, $, _, WebGMEGlobal*/
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Tue May 24 2016 10:15:19 GMT-0500 (CDT).
*/
define(['js/PanelBase/PanelBaseWithHeader',
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/ExecutionView/ExecutionViewWidget',
'./ExecutionViewControl'
], function (PanelBaseWithHeader,
IActivePanel,
ExecutionViewWidget,
ExecutionViewControl) {
], function (
PanelBaseWithHeader,
IActivePanel,
ExecutionViewWidget,
ExecutionViewControl
) {
'use strict';
var ExecutionViewPanel;
@@ -39,17 +39,24 @@ define(['js/PanelBase/PanelBaseWithHeader',
_.extend(ExecutionViewPanel.prototype, IActivePanel.prototype);
ExecutionViewPanel.prototype._initialize = function () {
var self = this;
var self = this,
footer = $('<div>', {class: 'footer-caption-container'});
this.$_el.append(footer);
//set Widget title
this.setTitle('');
this.widget = new ExecutionViewWidget(this.logger, this.$el);
this.widget.setTitle = function (title) {
this.widget._setTitle = function (title) {
self.setTitle(title);
};
this.widget.getFooterContainer = function () {
return footer;
};
this.control = new ExecutionViewControl({
logger: this.logger,
client: this._client,
@@ -57,6 +64,7 @@ define(['js/PanelBase/PanelBaseWithHeader',
widget: this.widget
});
footer.on('click', this.control.onOriginClicked.bind(this.control));
this.onActivate();
};
+83
Ver Arquivo
@@ -0,0 +1,83 @@
/*globals define, _, WebGMEGlobal, $ */
/*jshint browser: true*/
/**
* @author rkereskenyi / https://github.com/rkereskenyi
*/
define([
'js/PanelBase/PanelBase',
'js/Widgets/NetworkStatus/NetworkStatusWidget',
'js/Widgets/BranchStatus/BranchStatusWidget',
'js/Widgets/BranchSelector/BranchSelectorWidget',
'js/Widgets/KeyboardManager/KeyboardManagerWidget',
'js/Widgets/Notification/NotificationWidget'
], function (PanelBase,
NetworkStatusWidget,
BranchStatusWidget,
BranchSelectorWidget,
KeyboardManagerWidget,
NotificationWidget) {
'use strict';
var FooterPanel,
__parent__ = PanelBase;
FooterPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBase.OPTIONS.LOGGER_INSTANCE_NAME] = 'FooterPanel';
//call parent's constructor
__parent__.apply(this, [options]);
this._client = params.client;
//initialize UI
this._initialize();
this.logger.debug('FooterPanel ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(FooterPanel.prototype, __parent__.prototype);
FooterPanel.prototype._initialize = function () {
//main container
var navBar = $('<div/>', {class: 'navbar navbar-inverse navbar-fixed-bottom'}),
navBarInner = $('<div/>', {class: 'navbar-inner'}),
separator = $('<div class="spacer pull-right"></div>'),
widgetPlaceHolder = $('<div class="pull-right"></div>'),
keyBoardManagerEl,
networkStatusEl,
branchStatusEl,
notificationEl;
navBar.append(navBarInner);
this.$el.append(navBar);
//padding from screen right edge
navBarInner.append(separator.clone());
//keyboard enable/disbale widget (NOTE: only on non touch device)
if (WebGMEGlobal.SUPPORTS_TOUCH !== true) {
keyBoardManagerEl = widgetPlaceHolder.clone();
new KeyboardManagerWidget(keyBoardManagerEl);
navBarInner.append(keyBoardManagerEl).append(separator.clone());
}
networkStatusEl = widgetPlaceHolder.clone();
new NetworkStatusWidget(networkStatusEl, this._client);
navBarInner.append(networkStatusEl).append(separator.clone());
notificationEl = widgetPlaceHolder.clone();
new NotificationWidget(notificationEl, this._client);
navBarInner.append(notificationEl).append(separator.clone());
branchStatusEl = widgetPlaceHolder.clone();
new BranchStatusWidget(branchStatusEl, this._client);
navBarInner.append(branchStatusEl).append(separator.clone());
};
return FooterPanel;
});
+168 -120
Ver Arquivo
@@ -1,124 +1,66 @@
/*globals define, WebGMEGlobal*/
/*globals DeepForge, define, $, Materialize, WebGMEGlobal*/
// These are actions defined for specific meta types. They are evaluated from
// the context of the ForgeActionButton
define([
'q',
'js/RegistryKeys',
'js/Panels/MetaEditor/MetaEditorConstants',
'js/Constants'
'deepforge/globals'
], function(
REGISTRY_KEYS,
META_CONSTANTS,
CONSTANTS
Q,
REGISTRY_KEYS
) {
var instances = [
'Architecture',
'Pipeline'
],
metaNodes = [
'Operation',
'Data'
],
create = {};
var FILE_UPLOAD_INPUT = $('<input type="file" />');
var getUniqueName = function(parentId, basename) {
var pNode = this.client.getNode(parentId),
children = pNode.getChildrenIds().map(id => this.client.getNode(id)),
name = basename,
exists = {},
i = 2;
var createLayer = function() {
// Prompt the base type
this.promptLayerType().then(selected => {
var baseId = selected.node.id,
typeName = this.client.getNode(baseId).getAttribute('name'),
metanodes = this.client.getAllMetaNodes(),
msg = `Created new custom ${typeName} layer`,
newId,
customLayerId,
name;
children.forEach(child => exists[child.getAttribute('name')] = true);
for (var i = metanodes.length; i--;) {
name = metanodes[i].getAttribute('name');
if (name === 'CustomLayer') {
customLayerId = metanodes[i].getId();
break;
}
}
while (exists[name]) {
name = basename + '_' + i;
i++;
}
this.client.startTransaction(msg);
return name;
newId = this.createNamedNode(baseId, true);
this.addToMetaSheet(newId, 'CustomLayers');
this.client.addMixin(newId, customLayerId);
this.client.setRegistry(newId, REGISTRY_KEYS.IS_ABSTRACT, false);
this.client.completeTransaction();
WebGMEGlobal.State.registerActiveObject(newId);
});
};
var createNew = function(type, metasheetName) {
// Create CNN node in the current dir
// Get CNN node type
var parentId = this._currentNodeId,
newId,
baseId;
////////////// Downloading files //////////////
var downloadAttrs = [
'data',
'execFiles'
],
download = {};
baseId = this.client.getAllMetaNodes()
.find(node => node.getAttribute('name') === type)
.getId();
this.client.startTransaction('Created new operation prototype');
newId = this.client.createChild({parentId, baseId});
// Name the new node
var basename = 'New' + this.client.getNode(baseId).getAttribute('name'),
newName = getUniqueName.call(this, parentId, basename);
// If instance, make the first char lowercase
if (!metasheetName) {
newName = newName.substring(0, 1).toLowerCase() + newName.substring(1);
}
this.client.setAttributes(newId, 'name', newName);
if (metasheetName) { // Add to metasheet
var root = this.client.getNode(CONSTANTS.PROJECT_ROOT_ID),
metatabs = root.getRegistry(REGISTRY_KEYS.META_SHEETS),
metatab = metatabs.find(tab => tab.title === metasheetName) || metatabs[0],
metatabId = metatab.SetID;
// Add to the general meta
this.client.addMember(
CONSTANTS.PROJECT_ROOT_ID,
newId,
META_CONSTANTS.META_ASPECT_SET_NAME
);
this.client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
newId,
META_CONSTANTS.META_ASPECT_SET_NAME,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
// Add to the specific sheet
this.client.addMember(CONSTANTS.PROJECT_ROOT_ID, newId, metatabId);
this.client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
newId,
metatabId,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
}
this.client.completeTransaction();
WebGMEGlobal.State.registerActiveObject(newId);
};
instances.forEach(type => {
create[type] = function() {
return createNew.call(this, type);
};
});
metaNodes.forEach(type => {
create[type] = function() {
return createNew.call(this, type, type);
downloadAttrs.forEach(attr => {
download[attr] = function() {
return downloadButton.call(this, attr);
};
});
// Add download model button
var downloadButton = function() {
var downloadButton = function(attr) {
var id = this._currentNodeId,
node = this.client.getNode(id),
hash = node.getAttribute('data');
hash = node.getAttribute(attr);
if (hash) {
return '/rest/blob/download/' + hash;
@@ -126,41 +68,140 @@ define([
return null;
};
return {
Data: [
{
name: 'Download',
icon: 'play_for_work',
href: downloadButton // function to create href url
}
],
var importTorch = function() {
var pluginId = 'ImportTorch',
context = this.client.getCurrentPluginContext(pluginId),
fileInput = FILE_UPLOAD_INPUT.clone();
MyPipelines_META: [
// Prompt for the file
fileInput.on('change', event => this.uploadFile(event)
.then(hash => {
// Run the plugin in the browser (set namespace)
context.managerConfig.namespace = 'nn';
context.pluginConfig = {
srcHash: hash
};
return Q.ninvoke(this.client, 'runBrowserPlugin', pluginId, context);
})
.then(res => {
Materialize.toast(res.messages[0].message, 2000);
})
.fail(err => Materialize.toast(`Import failed: ${err}`, 2000))
);
fileInput.click();
};
var returnToLast = (place) => {
var returnId = DeepForge.last[place];
WebGMEGlobal.State.registerActiveObject(returnId);
};
var prototypeButtons = function(type, fromType) {
return [
{
name: 'Create new pipeline',
icon: 'queue',
action: create.Pipeline
name: `Return to ${fromType}`,
icon: 'input',
priority: 2,
filter: () => {
return DeepForge.last[fromType];
},
action: returnToLast.bind(null, fromType)
},
{
name: `Delete ${type} Definition`,
icon: 'delete',
priority: 1,
action: function() {
// Delete and go to the last pipeline?
var node = this.client.getNode(this._currentNodeId),
name = node.getAttribute('name'),
msg = `Deleted ${type} Definition for "${name}"`;
this.deleteCurrentNode(msg);
setTimeout(() => Materialize.toast(msg, 2000), 10);
returnToLast(fromType);
}
}
],
];
};
var MyPipelinesButtons = [
{
name: 'Create new pipeline',
icon: 'queue',
action: DeepForge.create.Pipeline
}
];
return {
HOME: MyPipelinesButtons,
MyPipelines_META: MyPipelinesButtons,
MyArchitectures_META: [
{
name: 'Create new architecture',
icon: 'queue',
action: create.Architecture
action: DeepForge.create.Architecture
},
{
name: 'Import Torch Architecture',
icon: 'swap_vert',
action: importTorch
}
],
MyDataTypes_META: [
{
name: 'Create new data type',
name: 'Create new primitive data type',
icon: 'queue',
action: create.Data
action: DeepForge.create.Primitive
},
{
name: 'Create new class',
icon: 'queue',
action: DeepForge.create.Complex
}
],
MyLayers_META: [
{
name: 'Create new layer',
icon: 'queue',
action: createLayer
}
],
MyOperations_META: [
{
name: 'Create new operation',
icon: 'queue',
action: create.Operation
action: DeepForge.create.Operation
}
],
MyArtifacts_META: [
{
name: 'Upload artifact',
icon: 'swap_vert',
action: DeepForge.create.Artifact
}
],
// Creating prototypes
Operation_META: prototypeButtons('Operation', 'Pipeline'),
Layer_META: prototypeButtons('Layer', 'Architecture'),
Complex_META: prototypeButtons('Class', 'Operation'),
Primitive_META: prototypeButtons('Primitive Type', 'Operation'),
// Instances
Data: [
{
name: 'Download',
icon: 'play_for_work',
href: download.data // function to create href url
}
],
Job: [
{
name: 'Download Execution Files',
icon: 'play_for_work',
href: download.execFiles
}
],
Pipeline: [
@@ -169,9 +210,16 @@ define([
icon: 'queue',
priority: 2,
action: function() {
this.onCreateInitialNode();
this.addOperation();
}
}
],
Architecture: [
{
name: 'Import Torch Architecture',
icon: 'swap_vert',
action: importTorch
}
]
};
});
@@ -1,26 +1,43 @@
/*globals define, _ */
/*globals DeepForge, $, define, _ */
/*jshint browser: true*/
define([
'blob/BlobClient',
'js/Constants',
'panel/FloatingActionButton/FloatingActionButton',
'deepforge/viz/PipelineControl',
'deepforge/viz/NodePrompter',
'./Actions',
'text!./PluginConfig.json'
'widgets/EasyDAG/AddNodeDialog',
'js/RegistryKeys',
'js/Panels/MetaEditor/MetaEditorConstants',
'q',
'text!./PluginConfig.json',
'deepforge/globals'
], function (
BlobClient,
CONSTANTS,
PluginButton,
PipelineControl,
NodePrompter,
ACTIONS,
AddNodeDialog,
REGISTRY_KEYS,
META_CONSTANTS,
Q,
PluginConfig
) {
'use strict';
var NEW_OPERATION_ID = '__NEW_OPERATION__';
var ForgeActionButton= function (layoutManager, params) {
PluginButton.call(this, layoutManager, params);
this._pluginConfig = JSON.parse(PluginConfig);
this._client = this.client;
this._actions = [];
this._blobClient = new BlobClient({
logger: this.logger.fork('BlobClient')
});
this.logger.debug('ctor finished');
};
@@ -37,14 +54,26 @@ define([
base = this.client.getNode(node.getMetaTypeId()),
isMeta = base && base.getId() === node.getId(),
suffix = isMeta ? '_META' : '',
actions,
basename;
while (base && !ACTIONS[basename]) {
basename = base.getAttribute('name') + suffix;
base = this.client.getNode(base.getBaseId());
if (!base) { // must be ROOT or FCO
basename = node.getAttribute('name') || 'ROOT_NODE';
actions = (ACTIONS[basename] || [])
.filter(action => !action.filter || action.filter());
return actions;
}
return ACTIONS[basename] || [];
while (base && !(actions && actions.length)) {
basename = base.getAttribute('name') + suffix;
base = this.client.getNode(base.getBaseId());
actions = ACTIONS[basename];
if (actions) {
actions = actions.filter(action => !action.filter || action.filter());
}
}
return actions || [];
};
ForgeActionButton.prototype.onNodeLoad = function(nodeId) {
@@ -70,5 +99,227 @@ define([
this.update();
};
// Helper functions REMOVE! FIXME
ForgeActionButton.prototype.addToMetaSheet = function(nodeId, metasheetName) {
var root = this.client.getNode(CONSTANTS.PROJECT_ROOT_ID),
metatabs = root.getRegistry(REGISTRY_KEYS.META_SHEETS),
metatab = metatabs.find(tab => tab.title === metasheetName) || metatabs[0],
metatabId = metatab.SetID;
// Add to the general meta
this.client.addMember(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
META_CONSTANTS.META_ASPECT_SET_NAME
);
this.client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
META_CONSTANTS.META_ASPECT_SET_NAME,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
// Add to the specific sheet
this.client.addMember(CONSTANTS.PROJECT_ROOT_ID, nodeId, metatabId);
this.client.setMemberRegistry(
CONSTANTS.PROJECT_ROOT_ID,
nodeId,
metatabId,
REGISTRY_KEYS.POSITION,
{
x: 100,
y: 100
}
);
};
ForgeActionButton.prototype.createNamedNode = function(baseId, isMeta) {
var parentId = this._currentNodeId,
newId = this.client.createChild({parentId, baseId}),
basename = 'New' + this.client.getNode(baseId).getAttribute('name'),
newName = this.getUniqueName(parentId, basename);
// If instance, make the first char lowercase
if (!isMeta) {
newName = newName.substring(0, 1).toLowerCase() + newName.substring(1);
}
this.client.setAttributes(newId, 'name', newName);
return newId;
};
ForgeActionButton.prototype.getUniqueName = function(parentId, basename) {
var pNode = this.client.getNode(parentId),
children = pNode.getChildrenIds().map(id => this.client.getNode(id)),
name = basename,
exists = {},
i = 2;
children.forEach(child => exists[child.getAttribute('name')] = true);
while (exists[name]) {
name = basename + '_' + i;
i++;
}
return name;
};
ForgeActionButton.prototype.getLayerTypeDesc = function(node) {
var decManager = this.client.decoratorManager,
desc = {};
desc.id = node.getId();
desc.name = node.getAttribute('name');
desc.baseName = desc.name;
desc.attributes = {};
desc.pointers = {};
// Get the decorator
desc.Decorator = decManager.getDecoratorForWidget('EllipseDecorator', 'EasyDAG');
// Set the color
desc.color = '#9e9e9e';
return desc;
};
ForgeActionButton.prototype.promptLayerType = function() {
// Prompt for the new custom layer's base type
var metanodes = this.client.getAllMetaNodes(),
baseLayerId = metanodes.find(n => n.getAttribute('name') === 'Layer').getId(),
layerType,
types;
// PoA:
// - Get the layer type ids
// - Create the descriptors
// - Get the color for the given types
// - Move colors to a constants dir?
// Get the layer type ids
layerType = metanodes
.filter(node => node.getBaseId() === baseLayerId);
// - Create the descriptors
types = layerType.map(node => {
return {
node: this.getLayerTypeDesc(node)
};
});
return AddNodeDialog.prompt(types);
};
ForgeActionButton.prototype.uploadFile = function(event) {
var deferred = Q.defer(),
file,
files,
afName,
artifact;
// cancel event and hover styling
event.stopPropagation();
event.preventDefault();
// fetch FileList object
files = event.target.files || event.dataTransfer.files;
// should only receive one file
if (files && files.length > 0) {
if (files.length > 1) {
this.logger.warn('Received multiple files. Using only the first');
}
afName = 'imported-architecture';
artifact = this._blobClient.createArtifact(afName);
file = files[0];
artifact.addFileAsSoftLink(file.name, file, (err, hash) => {
if (err) {
deferred.reject(err);
return;
}
deferred.resolve(hash);
});
}
return deferred.promise;
};
/////////////// Expanding containers ///////////////
ForgeActionButton.prototype.addOperation = function() {
var ops = this.getValidInitialNodes(),
newOperation = this.getNewOpNode();
// Add the 'New op button'
ops.push(newOperation);
this.promptNode(ops, (selected, prompter) => {
if (selected.id === NEW_OPERATION_ID) {
prompter.destroy();
DeepForge.create.Operation();
} else {
this.createNode(selected.id);
}
});
};
ForgeActionButton.prototype.getNewOpNode = function() {
var Decorator = this.client.decoratorManager.getDecoratorForWidget(
'OperationDecorator', 'EasyDAG');
return {
id: NEW_OPERATION_ID,
class: 'create-node',
name: 'New Operation...',
Decorator: Decorator,
attributes: {}
};
};
ForgeActionButton.prototype.promptNode = function(nodes, selectFn) {
// Get the absolute location of the given button
var mainBtn = this.$el[0].children[0],
rect = mainBtn.getBoundingClientRect(),
panelRect,
panelWidth = 400,
panelHeight = 400,
btns = this.$el.find('.tooltipped'),
ids;
this.$el.closeFAB();
// Hide the tooltip
ids = Array.prototype.map.call(btns, el => el.getAttribute('data-tooltip-id'));
ids.map(id => $('#' + id))
.filter(matches => matches.length)
.forEach(tooltip => tooltip.hide());
panelRect = {
left: rect.right-panelWidth,
top: rect.bottom-panelHeight,
width: panelWidth,
height: panelHeight
};
var cx = panelWidth-rect.width/2,
cy = panelHeight-rect.width/2,
prompter = new NodePrompter(panelRect, {cx, cy, padding: 5});
return prompter.prompt(nodes, selectFn);
};
ForgeActionButton.prototype.deleteCurrentNode = function(msg) {
var nodeId = this._currentNodeId;
if (nodeId) {
this.client.startTransaction(msg);
this.client.delMoreNodes([nodeId]);
this.client.completeTransaction(msg);
}
};
return ForgeActionButton;
});
@@ -10,5 +10,9 @@
"ImportTorch": {
"icon": "import_export",
"priority": -1
},
"GenerateExecFile": {
"icon": "play_for_work",
"priority": -1
}
}
@@ -0,0 +1,115 @@
/*globals define, _*/
/*jshint browser: true*/
define([
'panels/TextEditor/TextEditorControl'
], function (
TextEditorControl
) {
'use strict';
var NO_CODE_MESSAGE = '-- <%= name %> is not an editable layer!',
LayerEditorControl;
LayerEditorControl = function (options) {
TextEditorControl.call(this, options);
};
_.extend(LayerEditorControl.prototype, TextEditorControl.prototype);
LayerEditorControl.prototype.loadMetaNodes = function () {
return this._client.getAllMetaNodes();
};
// This next function retrieves the relevant node information for the widget
LayerEditorControl.prototype._getObjectDescriptor = function (nodeId) {
var desc = TextEditorControl.prototype._getObjectDescriptor.call(this, nodeId),
node = this._client.getNode(nodeId),
hasCode = node.getValidAttributeNames().indexOf('code') > -1,
template;
// Get own attribute, if set. Otherwise, set the text to the parent's populated
// template
this.loadMetaNodes();
if (hasCode) { // is a custom layer
if (!node.getOwnAttribute('code')) {
// Retrieve the template from the mixin
template = node.getMixinPaths()
.map(id => this._client.getNode(id).getAttribute('code'))
.find(code => !!code) || NO_CODE_MESSAGE;
}
} else {
template = NO_CODE_MESSAGE;
}
if (template) {
desc.text = _.template(template)(desc);
}
return desc;
};
LayerEditorControl.prototype.saveTextFor = function (id, text) {
var r = /:__init\((.*)\)/,
match = text.match(r),
textMatch = match && match[1],
node = this._client.getNode(id),
currentAttrs = node.getValidAttributeNames(),
attributes = [],
msg = `Updating layer definition for ${node.getAttribute('name')}`;
// Parse the attributes and update the node!
if (textMatch) {
attributes = textMatch.split(',')
.map(arg => arg.replace(/\s+/g, '')) // trim white space
.filter(arg => !!arg); // no empty strings!
} else { // inheriting __init
attributes = this.getInheritedAttrs(text);
}
this._client.startTransaction(msg);
TextEditorControl.prototype.saveTextFor.call(this, id, text);
// Remove old attributes
_.difference(currentAttrs, attributes)
.forEach(attr => this._client.removeAttributeSchema(id, attr));
attributes.forEach((attr, i) =>
this._client.setAttributeSchema(id, attr, {type: 'string', argindex: i}));
this._client.completeTransaction();
};
LayerEditorControl.prototype.getInheritedAttrs = function (code) {
// Get the base class
var r = /torch.class\((.*)\)/,
match = code.match(r),
baseType,
metanode,
textMatch = match && match[1];
if (textMatch) {
baseType = textMatch.split(',')[1]
.replace(/^\s*['"]nn\./, '')
.replace(/['"]\s*$/, '');
this._logger.debug(`inheriting the attributes from ${baseType}`);
// Get the meta node and valid attribute names
metanode = this._client.getAllMetaNodes()
.find(node => node.getAttribute('name') === baseType);
if (metanode) {
return metanode.getValidAttributeNames()
.filter(attr => attr !== 'name');
} else {
// Check if the type is known by torch
this._logger.warn(`Unknown base type ${baseType}. Assuming attributes are []`);
}
}
return [];
};
return LayerEditorControl;
});
@@ -0,0 +1,101 @@
/*globals define, _, WebGMEGlobal*/
/*jshint browser: true*/
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/TextEditor/TextEditorWidget',
'./LayerEditorControl'
], function (
PanelBaseWithHeader,
IActivePanel,
TextEditorWidget,
LayerEditorControl
) {
'use strict';
var LayerEditorPanel;
LayerEditorPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'LayerEditorPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
//initialize UI
this._initialize();
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(LayerEditorPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(LayerEditorPanel.prototype, IActivePanel.prototype);
LayerEditorPanel.prototype._initialize = function () {
var self = this;
//set Widget title
this.setTitle('');
this.widget = new TextEditorWidget(this.logger, this.$el);
this.widget.setTitle = function (title) {
self.setTitle(title);
};
this.control = new LayerEditorControl({
logger: this.logger,
client: this._client,
embedded: this._embedded,
widget: this.widget
});
this.onActivate();
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
LayerEditorPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
LayerEditorPanel.prototype.onResize = function (width, height) {
this.logger.debug('onResize --> width: ' + width + ', height: ' + height);
this.widget.onWidgetContainerResize(width, height);
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
LayerEditorPanel.prototype.destroy = function () {
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
LayerEditorPanel.prototype.onActivate = function () {
this.widget.onActivate();
this.control.onActivate();
WebGMEGlobal.KeyboardManager.setListener(this.widget);
WebGMEGlobal.Toolbar.refresh();
};
LayerEditorPanel.prototype.onDeactivate = function () {
this.widget.onDeactivate();
this.control.onDeactivate();
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
return LayerEditorPanel;
});
@@ -0,0 +1,29 @@
/*globals define, _*/
/*jshint browser: true*/
// This is a read-only view of the 'stdout' attribute for a Job node
define([
'panels/TextEditor/TextEditorControl'
], function (
TextEditorControl
) {
'use strict';
var LogViewerControl;
LogViewerControl = function (options) {
options.attributeName = 'stdout';
TextEditorControl.call(this, options);
};
_.extend(LogViewerControl.prototype, TextEditorControl.prototype);
LogViewerControl.prototype._onUpdate = function (id) {
if (id === this._currentNodeId) {
TextEditorControl.prototype._onUpdate.call(this, id);
}
};
return LogViewerControl;
});
@@ -0,0 +1,104 @@
/*globals define, _, WebGMEGlobal*/
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Wed Jun 15 2016 14:06:10 GMT-0500 (CDT).
*/
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/LogViewer/LogViewerWidget',
'./LogViewerControl'
], function (
PanelBaseWithHeader,
IActivePanel,
LogViewerWidget,
LogViewerControl
) {
'use strict';
var LogViewerPanel;
LogViewerPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'LogViewerPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
//initialize UI
this._initialize();
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(LogViewerPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(LogViewerPanel.prototype, IActivePanel.prototype);
LogViewerPanel.prototype._initialize = function () {
var self = this;
//set Widget title
this.setTitle('');
this.widget = new LogViewerWidget(this.logger, this.$el);
this.widget.setTitle = function (title) {
self.setTitle(title);
};
this.control = new LogViewerControl({
logger: this.logger,
client: this._client,
embedded: this._embedded,
widget: this.widget
});
this.onActivate();
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
LogViewerPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
LogViewerPanel.prototype.onResize = function (width, height) {
this.logger.debug('onResize --> width: ' + width + ', height: ' + height);
this.widget.onWidgetContainerResize(width, height);
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
LogViewerPanel.prototype.destroy = function () {
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
LogViewerPanel.prototype.onActivate = function () {
this.widget.onActivate();
this.control.onActivate();
WebGMEGlobal.KeyboardManager.setListener(this.widget);
WebGMEGlobal.Toolbar.refresh();
};
LogViewerPanel.prototype.onDeactivate = function () {
this.widget.onDeactivate();
this.control.onDeactivate();
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
return LogViewerPanel;
});
@@ -0,0 +1,225 @@
/*globals define, WebGMEGlobal*/
/*jshint browser: true*/
define([
'blob/BlobClient',
'js/Constants',
'js/Utils/GMEConcepts',
'js/NodePropertyNames',
'deepforge/globals'
], function (
BlobClient,
CONSTANTS,
GMEConcepts,
nodePropertyNames,
DeepForge
) {
'use strict';
var MainViewControl;
MainViewControl = function (options) {
this._logger = options.logger.fork('Control');
this._client = options.client;
// Initialize core collections and variables
this._widget = options.widget;
this._currentNodeId = null;
this._embedded = options.embedded;
this.territory = {};
this.ui = {};
this._blobClient = new BlobClient({
logger: this._logger.fork('BlobClient')
});
this._initWidgetEventHandlers();
this._logger.debug('ctor finished');
};
MainViewControl.prototype._initWidgetEventHandlers = function () {
this._widget.deleteNode = id => {
var node = this._client.getNode(id),
baseId = node.getBaseId(),
base = this._client.getNode(baseId),
baseName = base.getAttribute('name'),
name = node.getAttribute('name'),
msg = `Deleting ${baseName} "${name}"`;
this._client.startTransaction(msg);
this._client.delMoreNodes([id]);
this._client.completeTransaction();
};
this._widget.dataUrlFor = id => {
var node = this._client.getNode(id),
hash = node.getAttribute('data');
if (hash) {
return this._blobClient.getDownloadURL(hash);
} else {
return null;
}
};
};
/* * * * * * * * Visualizer content update callbacks * * * * * * * */
// One major concept here is with managing the territory. The territory
// defines the parts of the project that the visualizer is interested in
// (this allows the browser to then only load those relevant parts).
MainViewControl.prototype.selectedObjectChanged = function (nodeId) {
this._logger.debug('activeObject nodeId \'' + nodeId + '\'');
// Remove current territory patterns
this.clearTerritoryRules();
this._currentNodeId = nodeId;
if (typeof this._currentNodeId === 'string') {
var terrTypes = [
/* [type, root dir] */
['arch', 'MyArchitectures'],
['artifact', 'MyArtifacts']
];
terrTypes.forEach(pair => {
var type = pair[0],
dirname = pair[1];
// Update the territory
this.territory[type] = {};
this.territory[type][DeepForge.places[dirname]] = {children: 1};
this.ui[type] = this._client.addUI(this, this.handleEvents.bind(this, type));
this._client.updateTerritory(this.ui[type], this.territory[type]);
});
}
};
MainViewControl.prototype.handleEvents = function (type, events) {
var event;
// Remove the containing dir
events = events.filter(e => !this.territory[type][e.eid]);
this._logger.debug('_eventCallback \'' + i + '\' items');
for (var i = events.length; i--;) {
event = events[i];
switch (event.etype) {
case CONSTANTS.TERRITORY_EVENT_LOAD:
this.onLoad(type, event.eid);
break;
case CONSTANTS.TERRITORY_EVENT_UPDATE:
this._onUpdate(type, event.eid);
break;
case CONSTANTS.TERRITORY_EVENT_UNLOAD:
this._onUnload(event.eid);
break;
default:
break;
}
}
this._logger.debug('_eventCallback \'' + events.length + '\' items - DONE');
};
MainViewControl.prototype.onLoad = function(type, id) {
// Load a node of the given type
var desc = this._getObjectDescriptor(type, id);
if (type === 'arch') {
this._widget.addArch(desc);
} else { // artifacts
this._widget.addArtifact(desc);
}
};
// This next function retrieves the relevant node information for the widget
MainViewControl.prototype._getArtifactDesc = function (id) {
var node = this._client.getNode(id),
data = node.getAttribute('data'),
desc = this._getBasicDesc(id);
desc.data = data;
return desc;
};
MainViewControl.prototype._getArchDesc =
MainViewControl.prototype._getBasicDesc = function (id) {
var node = this._client.getNode(id);
return {
id: id,
name: node.getAttribute('name')
};
};
MainViewControl.prototype._getObjectDescriptor = function (type, id) {
return type === 'arch' ?
this._getArchDesc(id) :
this._getArtifactDesc(id);
};
/* * * * * * * * Node Event Handling * * * * * * * */
MainViewControl.prototype._onUpdate = function (type, gmeId) {
var description = this._getObjectDescriptor(type, gmeId);
this._widget.updateNode(description);
};
MainViewControl.prototype._onUnload = function (gmeId) {
this._widget.removeNode(gmeId);
};
MainViewControl.prototype._stateActiveObjectChanged = function (model, activeObjectId) {
if (this._currentNodeId !== activeObjectId) {
this.selectedObjectChanged(activeObjectId);
}
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
MainViewControl.prototype.destroy = function () {
this._detachClientEventListeners();
this.clearTerritoryRules();
};
MainViewControl.prototype._attachClientEventListeners = function () {
this._detachClientEventListeners();
if (!this._embedded) {
WebGMEGlobal.State.on('change:' + CONSTANTS.STATE_ACTIVE_OBJECT,
this._stateActiveObjectChanged, this);
}
};
MainViewControl.prototype._detachClientEventListeners = function () {
if (!this._embedded) {
WebGMEGlobal.State.off('change:' + CONSTANTS.STATE_ACTIVE_OBJECT,
this._stateActiveObjectChanged);
}
};
MainViewControl.prototype.onActivate = function () {
this._attachClientEventListeners();
if (typeof this._currentNodeId === 'string') {
WebGMEGlobal.State.registerSuppressVisualizerFromNode(true);
WebGMEGlobal.State.registerActiveObject(this._currentNodeId);
WebGMEGlobal.State.registerSuppressVisualizerFromNode(false);
}
};
MainViewControl.prototype.clearTerritoryRules = function () {
if (Object.keys(this.ui).length) {
Object.keys(this.ui).forEach(id =>
this._client.removeUI(this.ui[id]));
}
};
MainViewControl.prototype.onDeactivate = function () {
this._detachClientEventListeners();
};
return MainViewControl;
});
@@ -0,0 +1,128 @@
/*globals define, $, _, WebGMEGlobal*/
/*jshint browser: true*/
// The main panel shows the PipelineIndex w/ a bar on the left for viewing architectures
// and pipelines
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/MainView/MainViewWidget',
'./MainViewControl',
'panels/PipelineIndex/PipelineIndexPanel',
'deepforge/globals'
], function (
PanelBaseWithHeader,
IActivePanel,
MainViewWidget,
MainViewControl,
PipelineIndexPanel,
DeepForge
) {
'use strict';
var MainViewPanel;
MainViewPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'MainViewPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
//initialize UI
this.$nav = $('<div>', {id: 'nav-container'});
this.$el.css({padding: 0});
this.embeddedPanel = new PipelineIndexPanel(layoutManager, params);
this.$embedded = this.embeddedPanel.$el;
this.$embedded.addClass('embedded');
this.$el.append(this.$nav, this.$embedded);
this._initialize();
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(MainViewPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(MainViewPanel.prototype, IActivePanel.prototype);
MainViewPanel.prototype._initialize = function () {
//set Widget title
this.setTitle('');
this.widget = new MainViewWidget(this.logger, this.$nav);
this.control = new MainViewControl({
logger: this.logger,
client: this._client,
embedded: this._embedded,
widget: this.widget
});
var controlObjectChanged = this.control.selectedObjectChanged;
this.control.selectedObjectChanged = nodeId => {
this.embeddedPanel.control.selectedObjectChanged(DeepForge.places.MyPipelines);
return controlObjectChanged.call(this.control, nodeId);
};
this.onActivate();
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
MainViewPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
MainViewPanel.prototype.onResize = function (width, height) {
var navWidth,
embeddedWidth;
this.logger.debug('onResize --> width: ' + width + ', height: ' + height);
this.widget.onWidgetContainerResize(width, height);
navWidth = this.widget.width();
embeddedWidth = width-navWidth;
this.$embedded.css({
width: embeddedWidth,
height: height,
left: navWidth,
margin: 'inherit'
});
this.embeddedPanel.onResize(embeddedWidth, height);
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
MainViewPanel.prototype.destroy = function () {
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
MainViewPanel.prototype.onActivate = function () {
this.widget.onActivate();
this.control.onActivate();
WebGMEGlobal.KeyboardManager.setListener(this.widget);
WebGMEGlobal.Toolbar.refresh();
};
MainViewPanel.prototype.onDeactivate = function () {
this.widget.onDeactivate();
this.control.onDeactivate();
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
return MainViewPanel;
});
@@ -1,15 +1,21 @@
/*globals define, */
/*globals define, WebGMEGlobal*/
/*jshint browser: true*/
define([
'deepforge/globals',
'deepforge/viz/RenameablePanel',
'panels/TilingViz/TilingVizPanel',
'panels/OperationCodeEditor/OperationCodeEditorPanel',
'panels/OperationInterfaceEditor/OperationInterfaceEditorPanel',
'js/Constants',
'underscore'
], function (
DeepForge,
RenameablePanel,
TilingViz,
CodeEditor,
InterfaceEditor,
CONSTANTS,
_
) {
'use strict';
@@ -18,13 +24,66 @@ define([
OperationEditorPanel = function (layoutManager, params) {
TilingViz.call(this, layoutManager, params);
this.initialize();
};
OperationEditorPanel.prototype.initialize = function () {
this.territory = {};
this.territoryId = null;
this._currentNodeId = null;
this.control = this;
// Set the editable title on node change
this.initializeRenameable();
};
//inherit from TilingViz
_.extend(OperationEditorPanel.prototype, TilingViz.prototype);
_.extend(
OperationEditorPanel.prototype,
RenameablePanel.prototype,
TilingViz.prototype
);
OperationEditorPanel.prototype.selectedObjectChanged = function (id) {
this._currentNodeId = id;
DeepForge.last.Operation = id;
if (typeof this._currentNodeId === 'string') {
// Setup the territory
this.territory = {};
this.territory[this._currentNodeId] = {children: 0};
this.territoryId = this._client.addUI(this, this._eventCallback.bind(this));
this._client.updateTerritory(this.territoryId, this.territory);
}
TilingViz.prototype.selectedObjectChanged.call(this, id);
};
OperationEditorPanel.prototype._eventCallback = function (events) {
events = events.find(e => e.eid === this._currentNodeId);
this.updateTitle();
};
OperationEditorPanel.prototype.updateTitle = function () {
var id = this._currentNodeId,
node = this._client.getNode(id),
name = node && node.getAttribute('name');
this.setTitle(name || '');
};
OperationEditorPanel.prototype.getPanels = function () {
return [CodeEditor, InterfaceEditor];
return [InterfaceEditor, CodeEditor];
};
OperationEditorPanel.prototype.onDeactivate = function () {
WebGMEGlobal.State.off('change:' + CONSTANTS.STATE_ACTIVE_OBJECT,
this._stateActiveObjectChanged);
if (this.territoryId) {
this._client.removeUI(this.territoryId);
}
TilingViz.prototype.onDeactivate.call(this);
};
return OperationEditorPanel;
@@ -0,0 +1,5 @@
/*globals define*/
define({
PRIMITIVE: '#b0bec5',
COMPLEX: '#78909c'
});
@@ -1,6 +1,8 @@
/*globals define*/
define([
'./Colors'
], function(
COLORS
) {
'use strict';
var OperationInterfaceEditorEvents = function() {
@@ -9,6 +11,25 @@ define([
this._widget.addRefTo = this.addRefTo.bind(this);
this._widget.changePtrName = this.changePtrName.bind(this);
this._widget.removePtr = this.removePtr.bind(this);
this._widget.getCreationNode = this.getCreationNode.bind(this);
};
OperationInterfaceEditorEvents.prototype.getCreationNode = function(type, id) {
var typeName = type === 'Complex' ? 'Class' : 'Primitive',
Decorator = this._client.decoratorManager.getDecoratorForWidget(
this.DEFAULT_DECORATOR, 'EasyDAG');
return {
node: {
id: id,
class: 'create-node',
name: `New ${typeName}...`,
Decorator: Decorator,
color: COLORS[type.toUpperCase()],
isPrimitive: type === 'Primitive',
attributes: {}
}
};
};
OperationInterfaceEditorEvents.prototype.allValidReferences = function() {
@@ -19,7 +40,7 @@ define([
var notTypes = ['Data', 'Operation', 'Pipeline'];
return this._client.getAllMetaNodes()
.filter(node => {
var plugins = node.getRegistry('validPlugins');
var plugins = node.getOwnRegistry('validPlugins');
// Convention is enforced; if the plugin generates lua artifacts,
// it should be called `Generate`.. (something)
return plugins && plugins.indexOf('Generate') !== -1;
@@ -34,22 +55,28 @@ define([
});
};
OperationInterfaceEditorEvents.prototype.allDataTypeIds = function() {
return this.allDataTypes().map(node => node.getId());
OperationInterfaceEditorEvents.prototype.allDataTypeIds = function(incAbstract) {
return this.allDataTypes(incAbstract).map(node => node.getId());
};
OperationInterfaceEditorEvents.prototype.allDataTypes = function() {
OperationInterfaceEditorEvents.prototype.allDataTypes = function(incAbstract) {
return this._client.getAllMetaNodes()
.filter(node => this.hasMetaName(node.getId(), 'Data'));
.filter(node => this.hasMetaName(node.getId(), 'Data', incAbstract))
.filter(node => !node.isAbstract());
};
OperationInterfaceEditorEvents.prototype._getValidSuccessorNodes = function(nodeId) {
// Return all data types in the meta
OperationInterfaceEditorEvents.prototype.getValidSuccessors = function(nodeId, isInput) {
var dataTypeIds;
if (nodeId !== this._currentNodeId) {
return [];
}
return this.allDataTypeIds().map(id => {
// Return all data types in the meta
// If input, include abstract types
dataTypeIds = this.allDataTypeIds(isInput);
return dataTypeIds.map(id => {
return {
node: this._getObjectDescriptor(id)
};
@@ -9,14 +9,18 @@
define([
'panels/EasyDAG/EasyDAGControl',
'js/Constants',
'deepforge/lua',
'deepforge/viz/OperationControl',
'./OperationInterfaceEditorControl.EventHandlers',
'./Colors',
'underscore'
], function (
EasyDAGControl,
CONSTANTS,
luajs,
OperationControl,
OperationInterfaceEditorControlEvents,
COLORS,
_
) {
@@ -30,6 +34,8 @@ define([
OperationInterfaceEditorControlEvents.call(this);
this._connections = {};
this._pointers = {};
this._usage = {}; // info about input usage
};
_.extend(
@@ -139,18 +145,39 @@ define([
desc.container = cntr.toLowerCase();
desc.attributes = {};
} else if (desc.id === this._currentNodeId) {
desc.pointers = {};
delete desc.attributes.code;
}
// Extra decoration for data
if (this.hasMetaName(desc.id, 'Data', true)) {
desc.color = this.getDescColor(gmeId);
desc.isPrimitive = this.hasMetaName(gmeId, 'Primitive');
if (desc.container === 'inputs') {
desc.used = this.isUsedInCode(desc.name);
this._usage[desc.id] = desc.used;
} else {
desc.used = true;
}
}
return desc;
};
OperationInterfaceEditorControl.prototype.getDescColor = function(gmeId) {
return !this.hasMetaName(gmeId, 'Primitive', true) ? COLORS.COMPLEX :
COLORS.PRIMITIVE;
};
OperationInterfaceEditorControl.prototype._onUnload = function(gmeId) {
EasyDAGControl.prototype._onUnload.call(this, gmeId);
var conn = this._connections[gmeId];
if (conn) {
this._widget.removeNode(conn.id);
}
delete this._usage[gmeId];
};
OperationInterfaceEditorControl.prototype._onLoad = function(gmeId) {
@@ -170,12 +197,33 @@ define([
};
OperationInterfaceEditorControl.prototype._onUpdate = function(gmeId) {
var inputIds,
wasUsed,
isUsed,
name,
ast,
code;
if (gmeId === this._currentNodeId) {
EasyDAGControl.prototype._onUpdate.call(this, gmeId);
// Update the valid pointers
this.updatePtrs();
// Update the remaining usage info
inputIds = Object.keys(this._usage);
code = this._client.getNode(this._currentNodeId).getAttribute('code');
ast = luajs.parser.parse(code);
for (var i = inputIds.length; i--;) {
wasUsed = this._usage[inputIds[i]];
name = this._client.getNode(inputIds[i]).getAttribute('name');
isUsed = this.isUsedInCode(name, ast);
if (isUsed !== wasUsed) {
this._onUpdate(inputIds[i]);
}
}
} else if (this.containedInCurrent(gmeId) && this.hasMetaName(gmeId, 'Data')) {
EasyDAGControl.prototype._onUpdate.call(this, gmeId);
}
@@ -199,6 +247,7 @@ define([
isPointer: true,
baseName: target.getAttribute('name'),
Decorator: Decorator,
used: this.isUsedInCode(name),
attributes: {},
name: name,
parentId: this._currentNodeId
@@ -282,5 +331,39 @@ define([
return conn;
};
////////////////////// Unused input checking //////////////////////
OperationInterfaceEditorControl.prototype.isUsedInCode = function(name, ast) {
var code = this._client.getNode(this._currentNodeId).getAttribute('code'),
r = new RegExp('\\b' + name + '\\b'),
hasText = code.match(r) !== null;
// verify that it is not used only in the left side of an assignment
if (hasText) {
ast = ast || luajs.parser.parse(code);
return this.isUsedInNode(name, ast);
}
return false;
};
// Check if it is used in the given ast node
OperationInterfaceEditorControl.prototype.isUsedInNode = function(name, node) {
var isUsed = false,
checker;
checker = luajs.codegen.traverse((curr, parent) => {
if (curr.type === 'variable' && curr.val === name) {
// Ignore if it is being assigned...
if (parent.type === 'stat.assignment') {
return parent.right.indexOf(curr) !== -1;
}
return true;
}
}, found => isUsed = isUsed || (found === true));
checker(node);
return isUsed;
};
return OperationInterfaceEditorControl;
});
@@ -2,15 +2,19 @@
/*jshint browser: true*/
define([
'js/Constants',
'panels/EasyDAG/EasyDAGControl',
'deepforge/viz/PipelineControl',
'deepforge/globals',
'common/core/coreQ',
'common/storage/constants',
'q',
'underscore'
], function (
CONSTANTS,
EasyDAGControl,
PipelineControl,
DeepForge,
Core,
STORAGE_CONSTANTS,
Q,
@@ -23,11 +27,22 @@ define([
CONN = {
SRC: 'src',
DST: 'dst'
};
},
DECORATORS = {
ArtifactLoader: 'ArtifactOpDecorator',
ArtifactFinder: 'ArtifactOpDecorator'
},
WIDGET_NAME = 'EasyDAG';
PipelineEditorControl = function (options) {
EasyDAGControl.call(this, options);
this.addedIds = {};
this.executionTerritory = {};
this.executionUI = null;
this.invalidated = {};
this._widget.deleteNode = id => {
this._deleteNode(id);
};
};
_.extend(
@@ -41,6 +56,7 @@ define([
PipelineEditorControl.prototype.TERRITORY_RULE = {children: 3};
PipelineEditorControl.prototype.selectedObjectChanged = function (nodeId) {
DeepForge.last.Pipeline = nodeId;
this._logger.debug('activeObject nodeId \'' + nodeId + '\'');
// Remove current territory patterns
@@ -53,7 +69,7 @@ define([
if (typeof this._currentNodeId === 'string') {
var desc = this._getObjectDescriptor(nodeId);
this._widget.setTitle(desc.name.toUpperCase());
this._widget.setTitle(desc.name);
if (typeof desc.parentId === 'string') {
this.$btnModelHierarchyUp.show();
@@ -68,6 +84,19 @@ define([
}
};
PipelineEditorControl.prototype.hasCurrentNode = function () {
return typeof this._currentNodeId === 'string';
};
PipelineEditorControl.prototype.onNodeNameChanged = function (from, to) {
var msg = `Renaming pipeline "${from}" -> "${to}"`;
if (from !== to && !/^\s*$/.test(to)) {
this._client.startTransaction(msg);
this._client.setAttributes(this._currentNodeId, 'name', to);
this._client.completeTransaction();
}
};
PipelineEditorControl.prototype.updateTerritory = function() {
var nodeId = this._currentNodeId;
@@ -94,6 +123,11 @@ define([
// Add a rule for them
.forEach(opId => this._territories[opId] = this.TERRITORY_RULE);
// Add arch/artifact dir to the territory
// loading more than necessary.... can restrict it in the future
// if perf is a problem
this._territories[CONSTANTS.PROJECT_ROOT_ID] = {children: 2};
this._client.updateTerritory(this._territoryId, this._territories);
};
@@ -102,6 +136,8 @@ define([
this._widget.getExistingPortMatches = this.getExistingPortMatches.bind(this);
this._widget.createConnection = this.createConnection.bind(this);
this._widget.removeConnection = this.removeConnection.bind(this);
this._widget.getDecorator = this.getDecorator.bind(this);
this._widget.updateThumbnail = this.updateThumbnail.bind(this);
};
PipelineEditorControl.prototype.isContainedInActive = function (gmeId) {
@@ -115,16 +151,45 @@ define([
var desc = this._getObjectDescriptor(gmeId);
if (desc.parentId === this._currentNodeId) {
this.addedIds[desc.id] = true;
return EasyDAGControl.prototype._onLoad.call(this, gmeId);
} else if (this.isContainedInActive(desc.parentId) && desc.isDataPort) {
// Validate any connections
if (this.isValid(desc)) {
return EasyDAGControl.prototype._onLoad.call(this, gmeId);
}
} else if (desc.parentId !== null &&
this.isContainedInActive(desc.parentId) && desc.isDataPort) {
// port added!
this.addedIds[desc.id] = true;
this._widget.addPort(desc);
}
};
PipelineEditorControl.prototype.isValid = function (desc) {
// If it is a "dangling connection", remove it!
if (desc.isConnection) {
if (!(desc.src && desc.dst)) {
var node = this._client.getNode(this._currentNodeId),
name = node.getAttribute('name'),
msg = `Removing invalid connection ${desc.id} in "${name}"`;
this.invalidated[desc.id] = true;
this._client.startTransaction(msg);
this._client.delMoreNodes([desc.id]);
this._client.completeTransaction();
return false;
}
}
return true;
};
PipelineEditorControl.prototype._onUnload = function (gmeId) {
// Check if it has been added
if (this.invalidated[gmeId]) {
// No need to notify the widget; this was filtered bc it was
// an invalid connection
delete this.invalidated[gmeId];
return;
}
if(this.addedIds[gmeId]) {
delete this.addedIds[gmeId];
return EasyDAGControl.prototype._onUnload.call(this, gmeId);
@@ -142,6 +207,19 @@ define([
} // Ignore any other updates - ie, Inputs/Outputs containers
};
PipelineEditorControl.prototype._getNodeDecorator = function (nodeObj) {
var decoratorManager = this._client.decoratorManager,
decorator,
decoratorClass;
var base = this._client.getNode(nodeObj.getMetaTypeId()),
baseName = base && base.getAttribute('name');
decorator = DECORATORS[baseName] || this.DEFAULT_DECORATOR;
decoratorClass = decoratorManager.getDecoratorForWidget(decorator, WIDGET_NAME);
return decoratorClass;
};
// Override the getSuccessors method to look up successors by operations
// with input nodes of the selected node's output type (prioritize the
// valid nodes that are using an unused output type, if one exists, ow
@@ -418,5 +496,153 @@ define([
}
};
PipelineEditorControl.prototype._getTargetDirs = function (typeIds) {
// Find the directories containing these types
return this._client.getNode(CONSTANTS.PROJECT_ROOT_ID).getChildrenIds()
// No referencing data meta types
.filter(id => {
var cMeta = this._client.getChildrenMeta(id),
validChildIds;
if (!cMeta) {
return false;
}
validChildIds = cMeta.items.map(item => item.id);
for (var i = typeIds.length; i--;) {
if (validChildIds.indexOf(typeIds[i]) !== -1) {
return true;
}
}
return false;
});
};
////////////////////// Execution Support //////////////////////
PipelineEditorControl.prototype.onActiveNodeUpdate =
PipelineEditorControl.prototype.onActiveNodeLoad = function (gmeId) {
var node = this._client.getNode(gmeId),
executionIds = node.getMemberIds('executions'),
executionRule = {children: 0};
if (this.executionUI) {
this._client.removeUI(this, this.executionEvents.bind(this));
}
this.executionTerritory = {};
// Create a territory for the executions
executionIds.forEach(id => this.executionTerritory[id] = executionRule);
this.executionUI = this._client.addUI(this, this.executionEvents.bind(this));
this._client.updateTerritory(this.executionUI, this.executionTerritory);
};
PipelineEditorControl.prototype.executionEvents = function (events) {
var event;
// Handle events related to the associated executions
for (var i = events.length; i--;) {
event = events[i];
if (event.etype === CONSTANTS.TERRITORY_EVENT_LOAD) {
this.onExecLoad(event.eid);
} else if (event.etype === CONSTANTS.TERRITORY_EVENT_UPDATE) {
this.onExecUpdate(event.eid);
} else if (event.etype === CONSTANTS.TERRITORY_EVENT_UNLOAD) {
this.onExecUnload(event.eid);
}
}
};
PipelineEditorControl.prototype.getExecDesc = function (id) {
var node = this._client.getNode(id);
return {
id: id,
createdAt: node.getAttribute('createdAt'),
status: node.getAttribute('status'),
name: node.getAttribute('name')
};
};
PipelineEditorControl.prototype.onExecLoad = function (id) {
var desc = this.getExecDesc(id);
this._widget.addExecution(desc);
};
PipelineEditorControl.prototype.onExecUnload = function (id) {
this._widget.removeExecution(id);
};
PipelineEditorControl.prototype.onExecUpdate = function (id) {
var desc = this.getExecDesc(id);
this._widget.updateExecution(desc);
};
PipelineEditorControl.prototype._detachClientEventListeners = function () {
if (this.executionUI) {
this._client.removeUI(this, this.executionEvents.bind(this));
}
EasyDAGControl.prototype._detachClientEventListeners.call(this);
};
////////////////////// Execution Support END //////////////////////
PipelineEditorControl.prototype.getDecorator = function (nodeId) {
var node = this._client.getNode(nodeId);
if (node) {
return this._getNodeDecorator(node);
} else {
return this._client.decoratorManager.getDecoratorForWidget(
this.DEFAULT_DECORATOR, WIDGET_NAME);
}
};
PipelineEditorControl.prototype.updateThumbnail = function (svg) {
var node = this._client.getNode(this._currentNodeId),
name,
attrs,
currentThumbnail,
attrName = 'thumbnail',
msg;
if (node) { // may have been deleted
name = node.getAttribute('name');
attrs = node.getValidAttributeNames();
currentThumbnail = node.getAttribute(attrName);
msg = `Updating pipeline thumbnail for "${name}"`;
if (attrs.indexOf(attrName) > -1 && currentThumbnail !== svg) {
this._client.startTransaction(msg);
this._client.setAttributes(this._currentNodeId, attrName, svg);
this._client.completeTransaction();
}
}
};
////////////////////// Criterion Support //////////////////////
PipelineEditorControl.prototype._getValidTargetsFor = function (id, ptr) {
// Check if the pointer is a Criterion pointer -> if so, only show the meta types
// and the ones in the custom layer dir
var typeIds = this._client.getPointerMeta(id, ptr).items.map(item => item.id),
types = typeIds.map(id => this._client.getNode(id)),
criterion = types.find(node => node.getAttribute('name') === 'Criterion'),
items,
criterionId;
if (criterion) {
// Get all criterion types
criterionId = criterion.getId();
items = this._client.getAllMetaNodes().map(node => node.getId())
.filter(id => this._client.isTypeOf(id, criterionId));
return items.map(id => {
return {
node: this._getObjectDescriptor(id)
};
});
} else {
return EasyDAGControl.prototype._getValidTargetsFor.apply(this, arguments);
}
};
return PipelineEditorControl;
});
@@ -1,4 +1,4 @@
/*globals define, _, WebGMEGlobal*/
/*globals define, $, _, WebGMEGlobal*/
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Thu May 19 2016 14:04:47 GMT-0500 (CDT).
@@ -31,6 +31,7 @@ define(['js/PanelBase/PanelBaseWithHeader',
//initialize UI
this._initialize();
this.$el.addClass('pipeline-editor');
this.logger.debug('ctor finished');
};
@@ -39,12 +40,15 @@ define(['js/PanelBase/PanelBaseWithHeader',
_.extend(PipelineEditorPanel.prototype, IActivePanel.prototype);
PipelineEditorPanel.prototype._initialize = function () {
var self = this;
var self = this,
execCntr = $('<div>', {class: 'execution-container'});
//set Widget title
this.setTitle('');
this.$_el.append(execCntr);
this.$_el.addClass('pipeline-editor');
this.widget = new PipelineEditorWidget(this.logger, this.$el);
this.widget = new PipelineEditorWidget(this.logger, this.$el, execCntr);
this.widget.setTitle = function (title) {
self.setTitle(title);
@@ -57,9 +61,25 @@ define(['js/PanelBase/PanelBaseWithHeader',
widget: this.widget
});
// Editable pipeline name
this.$panelHeaderTitle.on('dblclick', () => this.editTitle());
this.onActivate();
};
PipelineEditorPanel.prototype.editTitle = function () {
if (this.control.hasCurrentNode()) {
this.$panelHeaderTitle.editInPlace({
css: {
'z-index': 1000
},
onChange: (oldValue, newValue) => {
this.control.onNodeNameChanged(oldValue, newValue);
}
});
}
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
PipelineEditorPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
@@ -0,0 +1,225 @@
/*globals define, WebGMEGlobal*/
/*jshint browser: true*/
define([
'js/Constants',
'js/NodePropertyNames'
], function (
CONSTANTS,
nodePropertyNames
) {
'use strict';
var PipelineIndexControl;
PipelineIndexControl = function (options) {
this._logger = options.logger.fork('Control');
this._client = options.client;
// Initialize core collections and variables
this._widget = options.widget;
this._currentNodeId = null;
this._embedded = options.embedded;
this._initWidgetEventHandlers();
this._logger.debug('ctor finished');
};
PipelineIndexControl.prototype._initWidgetEventHandlers = function () {
this._widget.deletePipeline = id => {
var node = this._client.getNode(id),
name = node.getAttribute('name'),
msg = `Deleting pipeline "${name}"`;
// Change the current active object
this._client.startTransaction(msg);
this._client.delMoreNodes([id]);
this._client.completeTransaction();
};
this._widget.setName = (id, name) => {
var oldName = this._client.getNode(id).getAttribute('name'),
msg = `Renaming Pipeline: "${oldName}" -> "${name}"`;
if (oldName !== name && !/^\s*$/.test(name)) {
this._client.startTransaction(msg);
this._client.setAttributes(id, 'name', name);
this._client.completeTransaction();
}
};
};
/* * * * * * * * Visualizer content update callbacks * * * * * * * */
// One major concept here is with managing the territory. The territory
// defines the parts of the project that the visualizer is interested in
// (this allows the browser to then only load those relevant parts).
PipelineIndexControl.prototype.selectedObjectChanged = function (nodeId) {
this._logger.debug('activeObject nodeId \'' + nodeId + '\'');
// Remove current territory patterns
if (this._currentNodeId) {
this._client.removeUI(this._territoryId);
}
this._currentNodeId = nodeId;
if (typeof this._currentNodeId === 'string') {
// Put new node's info into territory rules
this._selfPatterns = {};
this._selfPatterns[nodeId] = {children: 1};
this._territoryId = this._client.addUI(this, this._eventCallback.bind(this));
// Update the territory
this._client.updateTerritory(this._territoryId, this._selfPatterns);
}
};
// This next function retrieves the relevant node information for the widget
PipelineIndexControl.prototype._getObjectDescriptor = function (nodeId) {
var node = this._client.getNode(nodeId),
objDescriptor;
if (node) {
objDescriptor = {
id: undefined,
name: undefined,
parentId: undefined,
thumbnail: node.getAttribute('thumbnail'),
executionCount: node.getMemberIds('executions').length
};
objDescriptor.id = node.getId();
objDescriptor.name = node.getAttribute(nodePropertyNames.Attributes.name);
objDescriptor.parentId = node.getParentId();
}
return objDescriptor;
};
/* * * * * * * * Node Event Handling * * * * * * * */
PipelineIndexControl.prototype._eventCallback = function (events) {
var i = events ? events.length : 0,
event;
this._logger.debug('_eventCallback \'' + i + '\' items');
while (i--) {
event = events[i];
switch (event.etype) {
case CONSTANTS.TERRITORY_EVENT_LOAD:
this._onLoad(event.eid);
break;
case CONSTANTS.TERRITORY_EVENT_UPDATE:
this._onUpdate(event.eid);
break;
case CONSTANTS.TERRITORY_EVENT_UNLOAD:
this._onUnload(event.eid);
break;
default:
break;
}
}
this._logger.debug('_eventCallback \'' + events.length + '\' items - DONE');
};
PipelineIndexControl.prototype._onLoad = function (gmeId) {
if (gmeId !== this._currentNodeId) {
var description = this._getObjectDescriptor(gmeId);
this._widget.addNode(description);
}
};
PipelineIndexControl.prototype._onUpdate = function (gmeId) {
if (gmeId !== this._currentNodeId) {
var description = this._getObjectDescriptor(gmeId);
this._widget.updateNode(description);
}
};
PipelineIndexControl.prototype._onUnload = function (gmeId) {
this._widget.removeNode(gmeId);
};
PipelineIndexControl.prototype._stateActiveObjectChanged = function (model, activeObjectId) {
if (this._currentNodeId !== activeObjectId) {
this.selectedObjectChanged(activeObjectId);
}
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
PipelineIndexControl.prototype.destroy = function () {
this._detachClientEventListeners();
this._removeToolbarItems();
};
PipelineIndexControl.prototype._attachClientEventListeners = function () {
this._detachClientEventListeners();
if (!this._embedded) {
WebGMEGlobal.State.on('change:' + CONSTANTS.STATE_ACTIVE_OBJECT, this._stateActiveObjectChanged, this);
}
};
PipelineIndexControl.prototype._detachClientEventListeners = function () {
if (!this._embedded) {
WebGMEGlobal.State.off('change:' + CONSTANTS.STATE_ACTIVE_OBJECT, this._stateActiveObjectChanged);
}
};
PipelineIndexControl.prototype.onActivate = function () {
this._attachClientEventListeners();
this._displayToolbarItems();
if (typeof this._currentNodeId === 'string') {
WebGMEGlobal.State.registerSuppressVisualizerFromNode(true);
WebGMEGlobal.State.registerActiveObject(this._currentNodeId);
WebGMEGlobal.State.registerSuppressVisualizerFromNode(false);
}
};
PipelineIndexControl.prototype.onDeactivate = function () {
this._detachClientEventListeners();
this._hideToolbarItems();
};
/* * * * * * * * * * Updating the toolbar * * * * * * * * * */
PipelineIndexControl.prototype._displayToolbarItems = function () {
if (this._toolbarInitialized === true) {
for (var i = this._toolbarItems.length; i--;) {
this._toolbarItems[i].show();
}
} else {
this._initializeToolbar();
}
};
PipelineIndexControl.prototype._hideToolbarItems = function () {
if (this._toolbarInitialized === true) {
for (var i = this._toolbarItems.length; i--;) {
this._toolbarItems[i].hide();
}
}
};
PipelineIndexControl.prototype._removeToolbarItems = function () {
if (this._toolbarInitialized === true) {
for (var i = this._toolbarItems.length; i--;) {
this._toolbarItems[i].destroy();
}
}
};
PipelineIndexControl.prototype._initializeToolbar = function () {
this._toolbarItems = [];
this._toolbarInitialized = true;
};
return PipelineIndexControl;
});
@@ -0,0 +1,94 @@
/*globals define, _, WebGMEGlobal*/
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Wed Jun 29 2016 16:10:46 GMT-0500 (CDT).
*/
define(['js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/PipelineIndex/PipelineIndexWidget',
'./PipelineIndexControl'
], function (PanelBaseWithHeader,
IActivePanel,
PipelineIndexWidget,
PipelineIndexControl) {
'use strict';
var PipelineIndexPanel;
PipelineIndexPanel = function (layoutManager, params) {
var options = {};
//set properties from options
options[PanelBaseWithHeader.OPTIONS.LOGGER_INSTANCE_NAME] = 'PipelineIndexPanel';
options[PanelBaseWithHeader.OPTIONS.FLOATING_TITLE] = true;
//call parent's constructor
PanelBaseWithHeader.apply(this, [options, layoutManager]);
this._client = params.client;
this._embedded = params.embedded;
//initialize UI
this._initialize();
this.logger.debug('ctor finished');
};
//inherit from PanelBaseWithHeader
_.extend(PipelineIndexPanel.prototype, PanelBaseWithHeader.prototype);
_.extend(PipelineIndexPanel.prototype, IActivePanel.prototype);
PipelineIndexPanel.prototype._initialize = function () {
//set Widget title
this.setTitle('');
this.widget = new PipelineIndexWidget(this.logger, this.$el);
this.control = new PipelineIndexControl({
logger: this.logger,
client: this._client,
embedded: this._embedded,
widget: this.widget
});
this.onActivate();
};
/* OVERRIDE FROM WIDGET-WITH-HEADER */
/* METHOD CALLED WHEN THE WIDGET'S READ-ONLY PROPERTY CHANGES */
PipelineIndexPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
};
PipelineIndexPanel.prototype.onResize = function (width, height) {
this.logger.debug('onResize --> width: ' + width + ', height: ' + height);
};
/* * * * * * * * Visualizer life cycle callbacks * * * * * * * */
PipelineIndexPanel.prototype.destroy = function () {
this.control.destroy();
this.widget.destroy();
PanelBaseWithHeader.prototype.destroy.call(this);
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
PipelineIndexPanel.prototype.onActivate = function () {
this.widget.onActivate();
this.control.onActivate();
WebGMEGlobal.KeyboardManager.setListener(this.widget);
WebGMEGlobal.Toolbar.refresh();
};
PipelineIndexPanel.prototype.onDeactivate = function () {
this.widget.onDeactivate();
this.control.onDeactivate();
WebGMEGlobal.KeyboardManager.setListener(undefined);
WebGMEGlobal.Toolbar.refresh();
};
return PipelineIndexPanel;
});
@@ -16,6 +16,7 @@ define([
SerializeEditorControl = function (options) {
options.attributeName = 'serialize';
TextEditorControl.call(this, options);
this._widget.setName = this.setName.bind(this);
};
_.extend(
@@ -1,8 +1,5 @@
/*globals define, WebGMEGlobal*/
/*jshint browser: true*/
/**
* Generated by VisualizerGenerator 1.7.0 from webgme on Wed May 18 2016 08:58:20 GMT-0500 (CDT).
*/
define([
'js/Constants',
@@ -39,21 +36,31 @@ define([
};
TextEditorControl.prototype._initWidgetEventHandlers = function () {
// TODO: Add a way to navigate out of the current widget...
this._widget.saveTextFor = (id, text) => {
if (this._currentNodeHasAttr) {
this._client.setAttributes(id, this.ATTRIBUTE_NAME, text);
this.saveTextFor(id, text);
} else {
this._logger.warn(`Cannot save attribute ${this.ATTRIBUTE_NAME} ` +
`for ${id} - node doesn't have the given attribute!`);
}
};
this._widget.setName = this.setName.bind(this);
};
TextEditorControl.prototype.saveTextFor = function (id, text) {
this._client.setAttributes(id, this.ATTRIBUTE_NAME, text);
};
TextEditorControl.prototype.setName = function (name) {
var node = this._client.getNode(this._currentNodeId),
oldName = node.getAttribute('name'),
msg = `Renaming ${oldName} -> ${name}`;
this._client.startTransaction(msg);
this._client.setAttributes(this._currentNodeId, 'name', name);
this._client.completeTransaction();
};
/* * * * * * * * Visualizer content update callbacks * * * * * * * */
// One major concept here is with managing the territory. The territory
// defines the parts of the project that the visualizer is interested in
// (this allows the browser to then only load those relevant parts).
TextEditorControl.prototype.TERRITORY_RULE = {children: 0};
TextEditorControl.prototype.selectedObjectChanged = function (nodeId) {
var self = this;
@@ -67,14 +74,15 @@ define([
self._currentNodeId = nodeId;
self._currentNodeParentId = undefined;
self._currentNodeHasAttr = (typeof self._client.getNode(self._currentNodeId)
.getAttribute(self.ATTRIBUTE_NAME)) === 'string';
self._currentNodeHasAttr = false;
if (typeof self._currentNodeId === 'string') {
var parentId = this._getParentId(nodeId);
// Put new node's info into territory rules
self._selfPatterns = {};
//self._widget.setTitle(desc.name.toUpperCase());
self._currentNodeHasAttr = self._client.getNode(self._currentNodeId)
.getValidAttributeNames().indexOf(self.ATTRIBUTE_NAME) > -1;
if (typeof parentId === 'string') {
self.$btnModelHierarchyUp.show();
@@ -178,6 +186,10 @@ define([
TextEditorControl.prototype.destroy = function () {
this._detachClientEventListeners();
this._removeToolbarItems();
if (this._territoryId) {
this._client.removeUI(this._territoryId);
}
};
TextEditorControl.prototype._attachClientEventListeners = function () {
@@ -4,14 +4,17 @@
* Generated by VisualizerGenerator 1.7.0 from webgme on Wed May 18 2016 08:58:20 GMT-0500 (CDT).
*/
define(['js/PanelBase/PanelBaseWithHeader',
define([
'js/PanelBase/PanelBaseWithHeader',
'js/PanelManager/IActivePanel',
'widgets/TextEditor/TextEditorWidget',
'./TextEditorControl'
], function (PanelBaseWithHeader,
IActivePanel,
TextEditorWidget,
TextEditorControl) {
], function (
PanelBaseWithHeader,
IActivePanel,
TextEditorWidget,
TextEditorControl
) {
'use strict';
var TextEditorPanel;
@@ -65,7 +68,7 @@ define(['js/PanelBase/PanelBaseWithHeader',
TextEditorPanel.prototype.onReadOnlyChanged = function (isReadOnly) {
//apply parent's onReadOnlyChanged
PanelBaseWithHeader.prototype.onReadOnlyChanged.call(this, isReadOnly);
this.widget.setReadOnly(isReadOnly);
};
TextEditorPanel.prototype.onResize = function (width, height) {
@@ -57,8 +57,9 @@ define([
(model, nodeId) => this.selectedObjectChanged(nodeId)
);
}
this.$el.css({padding: 0});
if (panels.length !== 2) {
if (panels.length > 2) {
this.logger.error(`Unsupported number of panels (${panels.length})`);
}
@@ -92,7 +93,7 @@ define([
};
TilingVizPanel.prototype.onResize = function (width, height) {
var pwidth = width/2;
var pwidth = width/this.getPanels().length;
this._containers.forEach((c, i) => c.css({
width: pwidth,

Alguns arquivos não foram exibidos porque demasiados arquivos foram alterados neste diff Mostrar Mais