Comparar commits

...

84 Commits

Autor SHA1 Mensagem Data
John Shewchuk 71ab7cd12a Fix remaining PR issues except build issues 2019-09-01 10:08:10 -07:00
John Shewchuk d6f2ff7d65 Improvements to account managment and permissions 2019-08-31 09:10:23 -07:00
John Shewchuk f357dd389e Clean up rendering and debug output 2019-08-29 10:15:38 -04:00
John Shewchuk d10ad107fb Merge branch 'develop' of https://github.com/microsoft/VoTT into johnshew/login 2019-08-29 09:16:16 -04:00
John Shewchuk d2a7850e7a update typescript version 2019-08-28 16:42:40 -04:00
John Shewchuk c797463c31 fix property 2019-08-27 20:52:10 -07:00
John Shewchuk 74ea621b11 Fixes for PR comments
* Oauth2 names
* Use configured keys
* Add simple unit tests
2019-08-27 20:11:47 -07:00
John Shewchuk 12089b9b9e clean up jest options 2019-08-27 12:41:09 -07:00
John Shewchuk 6cd04b5158 address PR feedback
* remove settings.json
* add simple unit:test
2019-08-27 12:35:48 -07:00
Wallace Breza 90577ca8a0 Revert horizontal asset bar (#881)
The horizontal asset bar looks more visually appealing on the bottom and it also opens up more real estate for the main editing surface.

However, it also reduces the overall usability of the app. The vertical scrolling behavior on the moues wheel no longer works and the keyboard shortcut hot keys are inverted.

Let's revert this change for now until we are able to overcome these critical usability requirements.
2019-08-26 16:52:47 -07:00
John Shewchuk 6484f43ce1 APIs working and ready for integration 2019-08-25 15:50:55 -07:00
John Shewchuk 52a19d3d7d enable auth end-to-end
remove all server state
2019-08-24 22:29:28 -07:00
John Shewchuk 6b0a0d602a working 2019-08-24 19:50:39 -07:00
John Shewchuk 2e664da4be config 2019-08-24 19:34:13 -07:00
John Shewchuk a825d62edf Type updates and error checking 2019-08-24 12:07:51 -07:00
John Shewchuk 64753a9c43 basic edits 2019-08-24 11:44:55 -07:00
John Shewchuk 76a03f69dc Local changes not working yet 2019-08-24 11:30:36 -07:00
John Shewchuk 925655c6a5 Typed session 2019-08-23 18:38:54 -07:00
John Shewchuk 453b2772b3 remove lint 2019-08-23 17:20:20 -07:00
John Shewchuk 0504c0addd Prep for including identity in session 2019-08-23 16:41:58 -07:00
Luisa de Miranda 11c962b8e8 feat: add preview of filepaths to project settings source and target connection dropdowns (#877) 2019-08-23 16:20:55 -07:00
John Shewchuk 93cbcc61fe Enable types for npm cookies package 2019-08-23 16:20:17 -07:00
John Shewchuk ff33a23cef store user in session 2019-08-23 16:08:52 -07:00
My a4f9f652ac chore: pipeline for preview release (queue manually) (#879) 2019-08-23 15:58:49 -07:00
James Trott aa2d692db9 Updated logging and VS Code Debug Terminal 2019-08-23 14:07:29 -07:00
Elizabeth Halper 15881b7999 feat: move asset preview to horizontal (#876)
feat: change sidebar from vertical to horizontal

Moved the vertical asset previewer to be horizontal and at the bottom of the screen. This helps maximize the size of the picture being tagged when it is a landscape photo.

AB#860
2019-08-23 13:30:37 -07:00
John Shewchuk 601e19dc3a Merge branch 'johnshew/login' of https://github.com/microsoft/VoTT into johnshew/login 2019-08-23 13:28:21 -07:00
John Shewchuk e45c9e8a43 Remove server session dependency 2019-08-23 13:28:02 -07:00
John Shewchuk 1d875fe04c Remove server session dependency 2019-08-23 13:26:46 -07:00
John Shewchuk f620b2aa33 Adjust scope to read/wrote profile not email 2019-08-23 12:11:16 -07:00
hermanho 9d64f4aa0d fix: test asset distribution to include all tags on test/train split (#823)
* fix: test asset distribution to include all tags on test/train split

The test asset may not included all tags when export with test/train split option in current venison (2.1.0).

* Extract the same split logic into helper function

* Formatting

* Inverting if statement
2019-08-23 10:39:02 -07:00
John Shewchuk eb8d4b4a8b redirect support and clean up
* improve logging
* add .env.template
* move more to .env and config
2019-08-23 09:49:49 -07:00
John Shewchuk 56407823b9 get PORT from env 2019-08-22 17:16:28 -07:00
John Shewchuk 1f428b3e41 Merge branch 'johnshew/login' of https://github.com/microsoft/VoTT into johnshew/login 2019-08-22 17:02:08 -07:00
John Shewchuk 41b8214ae4 Listen on 80 for deployment to Azure 2019-08-22 17:01:21 -07:00
My Ho aa20c45723 fix start script 2019-08-22 15:57:42 -07:00
John Shewchuk 6d52386bc2 Disable linting 2019-08-22 15:22:17 -07:00
John Shewchuk ef766911ee Update deployment directories 2019-08-22 15:19:22 -07:00
John Shewchuk f2ae6de538 fix yml bug - extra pool 2019-08-22 15:08:13 -07:00
John Shewchuk 8137d1a1cd Update deployment 2019-08-22 15:03:48 -07:00
John Shewchuk e54a27655f Set up CI with Azure Pipelines
[skip ci]
2019-08-22 14:54:57 -07:00
John Shewchuk d8407839c3 rename yaml file 2019-08-22 14:53:49 -07:00
John Shewchuk bfa5829c42 Added yaml and fixed logging 2019-08-22 14:43:49 -07:00
Tanner Barlow c0201ca51a Revert "feat: move asset preview to horizontal (#870)" (#874)
This reverts commit 5e25dc5406.
2019-08-22 14:23:52 -07:00
John Shewchuk 666e2d0c56 Skeleton APIs running
* Updatres to express to serve static pages
* In memory connection management
2019-08-22 13:42:11 -07:00
John Shewchuk 6203d61587 Security and execution environment
* seperate out config into .env
* move views to public directory
2019-08-22 12:09:57 -07:00
Elizabeth Halper 5e25dc5406 feat: move asset preview to horizontal (#870)
feat: change sidebar from vertical to horizontal

Moved the vertical asset previewer to be horizontal and at the bottom of the screen. This helps maximize the size of the picture being tagged when it is a landscape photo.

AB#860
2019-08-22 10:59:24 -07:00
Tanner Barlow add4680e7b fix: Resolve UnhandledPromiseRejection in test (#859) 2019-08-22 10:43:38 -07:00
John Shewchuk 52042db1dc Merge branch 'master' of https://github.com/microsoft/VoTT into johnshew/login 2019-08-22 10:07:10 -07:00
John Shewchuk c2bb5e8f37 working server
* updated all node modules
* converted to express 4
* converted to typescript
* fixed all ts-lint issues
2019-08-22 10:06:41 -07:00
John Shewchuk 94830cc1f7 initial typescript server 2019-08-21 16:05:47 -07:00
John Shewchuk 586aebad04 port 2019-08-20 12:19:29 -07:00
Tanner Barlow 60ebb41540 Merge pull request #856 from microsoft/master
release: Master into dev
2019-08-20 12:02:42 -07:00
Wallace Breza 745e854cc4 Release 2.1.0 (#790)
Updates package version and changelog for 2.1.0 release
2019-04-29 14:39:24 -07:00
Wallace Breza 2234c8a0cc fix: Updates backwards compat & fixes cntk export image bug (#789)
Fixes an issue where the images exported out of a video file were missing file extension for video projects.
2019-04-29 14:18:45 -07:00
Wallace Breza 4d02db4215 fix: Updates export options for pascalVOC rename (#788)
Adds a check during project load to update the export options if project was using previous pascalVOC name.
2019-04-29 14:18:45 -07:00
Lee, Jebum 90754dc74b fix: change method for alloc string to buffer (#777)
String.length is not appropriate for calculating buffer size
when non-alphabet letter is included in content.
Change the method Buffer.alloc to Buffer.from as directed by the nodejs document.
2019-04-29 14:18:45 -07:00
Jacopo Mangiavacchi f29963c89e feat: Add CSV Exporter (#757)
Adds CSV export provider
2019-04-29 14:18:45 -07:00
Tanner Barlow acbbc86151 fix: Fix display of tag color picker (#782)
Resolves issue of tag color picker not being shown on alt-click or color-click + edit button. Also adds several tests for increased test coverage of tagInput.tsx
2019-04-29 14:18:45 -07:00
Wallace Breza 921dbac155 feat: Active Learning Updates (#778)
Adds new active learning form
Moves active learning settings from project settings to here
Refactored and created activeLearningService
2019-04-29 14:18:45 -07:00
P.J. Little a2ef52c7a4 docs: updates to readme and changelog (#781)
Minor updates and corrections to the main readme and changelog.
2019-04-29 14:18:45 -07:00
Wallace Breza 25b4aa2dc8 Create CODE_OF_CONDUCT.md (#779)
Adds code of conduct
2019-04-29 14:18:45 -07:00
Wallace Breza 0429590bec doc: Add bug & feature templates (#780)
Adds bug and feature github templates
2019-04-29 14:18:45 -07:00
Wallace Breza 48805dcb85 test: Verify tag update/delete project actions 2019-04-29 14:18:45 -07:00
Wallace Breza 0b06d6ac5b test: Refactored editor page tests 2019-04-29 14:18:45 -07:00
Wallace Breza 3998b6efc8 fix: Refactored project tag/delete updates 2019-04-29 14:18:45 -07:00
Tanner Barlow 4a0dcb2905 Dummy commit to kick off build again 2019-04-29 14:18:45 -07:00
Tanner Barlow 354623ec21 Lint fixes 2019-04-29 14:18:45 -07:00
Tanner Barlow 8b34db5724 Clean up and docs 2019-04-29 14:18:45 -07:00
Tanner Barlow bbd83a4df5 Fix tag removal test for editor page 2019-04-29 14:18:45 -07:00
Tanner Barlow 5b4610b3d9 Rename tag function in editor page 2019-04-29 14:18:45 -07:00
Tanner Barlow 996a555333 Delete tag working 2019-04-29 14:18:45 -07:00
Tanner Barlow 8439574dc5 Saving assets in async foreach loop 2019-04-29 14:18:45 -07:00
Tanner Barlow 4193bc0e6a Register mixins and run async loop 2019-04-29 14:18:45 -07:00
Tanner Barlow f394ea3d10 Editor Page and canvas upates 2019-04-29 14:18:45 -07:00
Tanner Barlow 4f325dfe4b Split out project functions and asset functions into respective services 2019-04-29 14:18:45 -07:00
Tanner Barlow 2ef4e1387f Added confirm and strings for tag and rename operations 2019-04-29 14:18:45 -07:00
Tanner Barlow 1001528a16 Added tests for project service 2019-04-29 14:18:45 -07:00
Tanner Barlow 6974aef9d1 Project service functions 2019-04-29 14:18:45 -07:00
Tanner Barlow 37234ec2e9 refactor: Remove editor footer 2019-04-29 14:18:45 -07:00
Wallace Breza d6a059447d Fix: Enables selection of azure region for custom vision export (#765)
Adds ability to select azure region where your custom vision service is hosted.
Filters domain list by project type

Resolves #759, #770
2019-04-29 14:18:45 -07:00
Wallace Breza c10c971caf feat: CNTK Export Provider (#771)
Adds CNTK export provider into v2

Resolves #754
2019-04-29 14:18:45 -07:00
Wallace Breza 0fe63863b1 feat: Save partial project progress during project creation (#769)
This adds functionality to persist partial project information when creating a new project. Right now when creating a new connection inline within the create project flow and returning to the create project screen your partial project information is lost. Partial form progress is now saved into local storage and bound when returning to the form.

Resolves #758
2019-04-29 14:18:45 -07:00
Jacopo Mangiavacchi 39521f2b61 Fix ymax and rename Tensorflow nama everywhere (#763)
fix issue #760 [AB#18223]
2019-04-29 14:18:45 -07:00
38 arquivos alterados com 8813 adições e 75 exclusões
+6
Ver Arquivo
@@ -19,6 +19,7 @@
# misc
.DS_Store
.env
.env.local
.env.development.local
.env.test.local
@@ -37,3 +38,8 @@ secrets.sh
# complexity reports
es6-src/
report/
# VoTT Server
server/lib
server/node_modules
server/coverage
+18
Ver Arquivo
@@ -2,6 +2,24 @@
<!-- cl-start -->
# [2.1.0](https://github.com/Microsoft/VoTT/compare/v2.0.0...v2.1.0) (04-29-2019)
[GitHub Release](https://github.com/Microsoft/VoTT/releases/tag/v2.1.0)
- fix: Updates backwards compat & fixes cntk export image bug (#789)
- fix: Updates export options for pascalVOC rename (#788)
- fix: change method for alloc string to buffer (#777)
- feat: Add CSV Exporter (#757)
- fix: Fix display of tag color picker (#782)
- feat: Active Learning Updates (#778)
- doc: updates to readme and changelog (#781)
- doc: Adds CODE_OF_CONDUCT.md (#779)
- doc: Add bug & feature templates (#780)
- fix: Refactored project tag/delete updates (#764)
- fix: Enables selection of azure region for custom vision export (#765)
- feat: CNTK Export Provider (#771)
- feat: Save partial project progress during project creation (#769)
- fix: Fixes ymax and rename Tensorflow nama everywhere (#763)
# [2.0.0](https://github.com/Microsoft/VoTT/compare/v2.0.0-preview.3...v2.0.0) (04-12-2019)
[GitHub Release](https://github.com/Microsoft/VoTT/releases/tag/v2.0.0)
+1
Ver Arquivo
@@ -28,6 +28,7 @@ VoTT helps facilitate an end-to-end machine learning pipeline:
<!-- toc -->
- [VoTT (Visual Object Tagging Tool)](#vott-visual-object-tagging-tool)
- [Table of Contents](#table-of-contents)
- [Getting Started](#getting-started)
+28
Ver Arquivo
@@ -0,0 +1,28 @@
# https://docs.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops#job-templates-with-parameters
jobs:
- job: ${{ parameters.name }}
pool: ${{ parameters.pool }}
timeoutInMinutes: 15 # how long to run the job before automatically cancelling
steps:
- task: NodeTool@0
displayName: 'Use Node 10.x'
inputs:
versionSpec: 10.x
- bash: |
set -ex
# clean install
npm ci
npm run release-ci
OS=${{ parameters.os }}
ARTIFACT_NAME=${{ parameters.artifact }}
mkdir -p ${OS}
cp releases/${ARTIFACT_NAME} ${OS}/
displayName: Build
- publish: $(System.DefaultWorkingDirectory)/${{ parameters.os }}
artifact: ${{ parameters.os }}
@@ -0,0 +1,32 @@
parameters:
GitHubConnection: '' # defaults for any parameters that aren't specified
repositoryName: ''
releaseNotesSource: input
addChangeLog: false
isPreRelease: true
isDraft: true
jobs:
- job: Create_Github_Release
timeoutInMinutes: 30 # timeout on job if deploy is not completed in 30 minutes
pool:
vmImage: ubuntu-16.04
steps:
- checkout: none # we already have the artifacts built, don't need the code
- download: current
- task: GitHubRelease@0
displayName: 'GitHub release (create)'
inputs:
gitHubConnection: ${{ parameters.GitHubConnection }}
repositoryName: ${{ parameters.repositoryName }}
releaseNotesSource: ${{ parameters.releaseNotesSource }}
target: $(Build.SourceBranch)
assets: |
../linux/*
../windows/*
../mac/*
addChangeLog: ${{ parameters.addChangeLog }}
isDraft: true # for testing, change to true when ready to merge
isPreRelease: ${{ parameters.isPrelease }}
+21 -32
Ver Arquivo
@@ -1,6 +1,6 @@
{
"name": "vott",
"version": "2.0.0",
"version": "2.1.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
@@ -6253,6 +6253,14 @@
}
}
},
"express-request-id": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/express-request-id/-/express-request-id-1.4.1.tgz",
"integrity": "sha512-qpxK6XhDYtdx9FvxwCHkUeZVWtkGbWR87hBAzGECfwYF/QQCPXEwwB2/9NGkOR1tT7/aLs9mma3CT0vjSzuZVw==",
"requires": {
"uuid": "^3.3.2"
}
},
"extend": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
@@ -7319,8 +7327,7 @@
"ansi-regex": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
"integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8=",
"optional": true
"integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8="
},
"aproba": {
"version": "1.2.0",
@@ -7341,14 +7348,12 @@
"balanced-match": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.0.tgz",
"integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c=",
"optional": true
"integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c="
},
"brace-expansion": {
"version": "1.1.11",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
"integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
"optional": true,
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -7363,20 +7368,17 @@
"code-point-at": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/code-point-at/-/code-point-at-1.1.0.tgz",
"integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c=",
"optional": true
"integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c="
},
"concat-map": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
"integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=",
"optional": true
"integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s="
},
"console-control-strings": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/console-control-strings/-/console-control-strings-1.1.0.tgz",
"integrity": "sha1-PXz0Rk22RG6mRL9LOVB/mFEAjo4=",
"optional": true
"integrity": "sha1-PXz0Rk22RG6mRL9LOVB/mFEAjo4="
},
"core-util-is": {
"version": "1.0.2",
@@ -7493,8 +7495,7 @@
"inherits": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=",
"optional": true
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
},
"ini": {
"version": "1.3.5",
@@ -7506,7 +7507,6 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-1.0.0.tgz",
"integrity": "sha1-754xOG8DGn8NZDr4L95QxFfvAMs=",
"optional": true,
"requires": {
"number-is-nan": "^1.0.0"
}
@@ -7521,7 +7521,6 @@
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz",
"integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==",
"optional": true,
"requires": {
"brace-expansion": "^1.1.7"
}
@@ -7529,14 +7528,12 @@
"minimist": {
"version": "0.0.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz",
"integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0=",
"optional": true
"integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0="
},
"minipass": {
"version": "2.2.4",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-2.2.4.tgz",
"integrity": "sha512-hzXIWWet/BzWhYs2b+u7dRHlruXhwdgvlTMDKC6Cb1U7ps6Ac6yQlR39xsbjWJE377YTCtKwIXIpJ5oP+j5y8g==",
"optional": true,
"requires": {
"safe-buffer": "^5.1.1",
"yallist": "^3.0.0"
@@ -7555,7 +7552,6 @@
"version": "0.5.1",
"resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz",
"integrity": "sha1-MAV0OOrGz3+MR2fzhkjWaX11yQM=",
"optional": true,
"requires": {
"minimist": "0.0.8"
}
@@ -7636,8 +7632,7 @@
"number-is-nan": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz",
"integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0=",
"optional": true
"integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0="
},
"object-assign": {
"version": "4.1.1",
@@ -7649,7 +7644,6 @@
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha1-WDsap3WWHUsROsF9nFC6753Xa9E=",
"optional": true,
"requires": {
"wrappy": "1"
}
@@ -7735,8 +7729,7 @@
"safe-buffer": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg==",
"optional": true
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
},
"safer-buffer": {
"version": "2.1.2",
@@ -7772,7 +7765,6 @@
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-1.0.2.tgz",
"integrity": "sha1-EYvfW4zcUaKn5w0hHgfisLmxB9M=",
"optional": true,
"requires": {
"code-point-at": "^1.0.0",
"is-fullwidth-code-point": "^1.0.0",
@@ -7792,7 +7784,6 @@
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz",
"integrity": "sha1-ajhfuIU9lS1f8F0Oiq+UJ43GPc8=",
"optional": true,
"requires": {
"ansi-regex": "^2.0.0"
}
@@ -7836,14 +7827,12 @@
"wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=",
"optional": true
"integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8="
},
"yallist": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.0.2.tgz",
"integrity": "sha1-hFK0u36Dx8GI2AQcGoN8dz1ti7k=",
"optional": true
"integrity": "sha1-hFK0u36Dx8GI2AQcGoN8dz1ti7k="
}
}
},
@@ -14828,7 +14817,7 @@
"react-modal": {
"version": "3.8.1",
"resolved": "https://registry.npmjs.org/react-modal/-/react-modal-3.8.1.tgz",
"integrity": "sha1-cwD5Sm+SouF5lN4L5sy2FzRGTJ4=",
"integrity": "sha512-aLKeZM9pgXpIKVwopRHMuvqKWiBajkqisDA8UzocdCF6S4fyKVfLWmZR5G1Q0ODBxxxxf2XIwiCP8G/11GJAuw==",
"requires": {
"exenv": "^1.2.0",
"prop-types": "^15.5.10",
+2 -1
Ver Arquivo
@@ -1,6 +1,6 @@
{
"name": "vott",
"version": "2.0.0",
"version": "2.1.0",
"author": {
"name": "Microsoft",
"url": "https://github.com/Microsoft/VoTT"
@@ -23,6 +23,7 @@
"buffer-reverse": "^1.0.1",
"crypto-js": "^3.1.9-1",
"dotenv": "^7.0.0",
"express-request-id": "^1.4.1",
"google-protobuf": "^3.6.1",
"jpeg-js": "^0.3.4",
"json2csv": "^4.5.0",
+88
Ver Arquivo
@@ -0,0 +1,88 @@
trigger: none # manual queue only when we're ready to release
pr: none # disable CI build for PR
stages:
- stage: version_bump_commit
jobs:
- job: "version_bump"
variables:
- group: GitHub-Deploy-Creds
timeoutInMinutes: 30 # timeout on job if deploy is not completed in 30 minutes
cancelTimeoutInMinutes: 1 # time limit to wait for job to cancel
pool:
vmImage: macOS-10.13 # ssh key was generated on a Mac so using the same type of OS here
steps:
- task: NodeTool@0
inputs:
versionSpec: 10.x
displayName: 'Install Node.js'
# Download secure file
# Download a secure file to the agent machine
- task: DownloadSecureFile@1
# name: sshKey # The name with which to reference the secure file's path on the agent, like $(mySecureFile.secureFilePath)
inputs:
secureFile: vott_id_rsa
# Install an SSH key prior to a build or deployment
- task: InstallSSHKey@0 # https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/install-ssh-key?view=azure-devops
inputs:
knownHostsEntry: $(KNOWN_HOSTS_ENTRY)
sshPublicKey: $(SSH_PUBLIC_KEY)
#sshPassphrase: # Optional
sshKeySecureFile: vott_id_rsa
env:
KNOWN_HOSTS_ENTRY: $(KNOWN_HOSTS_ENTRY)
SSH_PUBLIC_KEY: $(SSH_PUBLIC_KEY) # map to the right format (camelCase) that Azure credentials understand
- task: Bash@3
name: BumpNpmVersion
displayName: Bump NPM Prerelease Version
inputs:
targetType: filePath
filePath: ./scripts/version-bump-commit.sh
env:
SOURCE_BRANCH: $(Build.SourceBranch)
- stage: package_build
dependsOn: version_bump_commit
jobs:
- template: azure-pipelines/templates/build-artifact.yml
parameters:
name: Linux
pool:
vmImage: ubuntu-16.04
os: linux
artifact: vott*.snap
- template: azure-pipelines/templates/build-artifact.yml
parameters:
name: Windows
pool:
vmImage: vs2017-win2016
os: windows
artifact: vott*.exe
- template: azure-pipelines/templates/build-artifact.yml
parameters:
name: MacOS
pool:
vmImage: macOS-10.13
os: mac
artifact: vott*.dmg
- stage: github_release
dependsOn: package_build
jobs:
- template: azure-pipelines/templates/create-github-release.yml
parameters:
GitHubConnection: 'GitHub connection' # defaults for any parameters that aren't specified
repositoryName: 'Microsoft/VoTT'
releaseNotesSource: input
addChangeLog: false
isPreRelease: true
isDraft: false
+28
Ver Arquivo
@@ -0,0 +1,28 @@
#!/bin/bash
set -euo pipefail
NPM_RELEASE_TYPE=${1-"prepatch --preid=preview"}
# Get full branch name excluding refs/head from the env var SOURCE_BRANCH
SOURCE_BRANCH_NAME=${SOURCE_BRANCH/refs\/heads\/}
# Configure git to commit as SLS Azure Functions Service Account
echo "Configuring git to use deploy key..."
git config --local user.email "vott@microsoft.com"
git config --local user.name "Vott"
echo "SOURCE_BRANCH: ${SOURCE_BRANCH_NAME}"
git pull origin ${SOURCE_BRANCH_NAME}
git checkout ${SOURCE_BRANCH_NAME}
echo "Checked out branch: ${SOURCE_BRANCH_NAME}"
NPM_VERSION=`npm version ${NPM_RELEASE_TYPE} -m "release: Update ${NPM_RELEASE_TYPE} version to %s ***NO_CI***"`
echo "Set NPM version to: ${NPM_VERSION}"
SHA=`git rev-parse HEAD`
export GIT_SSH_COMMAND="ssh -vvv -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
git remote add authOrigin git@github.com:microsoft/VoTT.git
git push authOrigin ${SOURCE_BRANCH_NAME} --tags
echo "Pushed new tag: ${NPM_VERSION} @ SHA: ${SHA:0:8}"
+5
Ver Arquivo
@@ -0,0 +1,5 @@
APP_ID=xyz
APP_SECRET=asdf
COOKIE_SECRETS="[ { key: '12345678901234567890123456789012', iv: '123456789012' }, { key: 'abcdefghijklmnopqrstuvwxyzabcdef', iv: 'abcdefghijkl' }, ])"
ALLOW_HTTP=true
BASE_URL=http://localhost:3000/
+35
Ver Arquivo
@@ -0,0 +1,35 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "launch",
"name": "Launch via NPM",
"runtimeExecutable": "npm",
"runtimeArgs": [
"run-script",
"debug"
],
"port": 9229
},
{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceFolder}/lib/app.js", //"${workspaceFolder}\\lib\\app.js",
"args": [
"|",
"bunyan"
],
"outFiles": [
"${workspaceFolder}/**/*.js"
],
"console": "internalConsole",
"outputCapture": "std",
}
]
}
+24
Ver Arquivo
@@ -0,0 +1,24 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=733558
// for the documentation about the tasks.json format
"version": "2.0.0",
"tasks": [
{
"type": "npm",
"script": "build",
"problemMatcher": [
"$tsc"
],
"group": "build"
},
{
"type": "typescript",
"tsconfig": "tsconfig.json",
"option": "watch",
"problemMatcher": [
"$tsc-watch"
],
"group": "build"
}
]
}
+75
Ver Arquivo
@@ -0,0 +1,75 @@
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- johnshew/login
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'fe7b93fe-e836-4a55-804c-883dbea6af24'
# Web app name
webAppName: 'vott'
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool@0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build --if-present
# npm run test --if-present
workingDirectory: $(System.DefaultWorkingDirectory)/server
displayName: 'npm install, build and test'
- task: ArchiveFiles@2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/server'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp@1
displayName: 'Azure Web App Deploy: vott'
inputs:
azureSubscription: $(azureSubscription)
appType: webAppLinux
appName: $(webAppName)
runtimeStack: 'NODE|10.10'
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'npm run start'
+6
Ver Arquivo
@@ -0,0 +1,6 @@
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
rootDir: "src",
coverageDirectory: "../coverage",
};
+7320
Ver Arquivo
Diferenças do arquivo suprimidas por serem muito extensas Carregar Diff
+62
Ver Arquivo
@@ -0,0 +1,62 @@
{
"name": "vott-server",
"version": "1.0.0",
"description": "Server to support VoTT with login",
"main": "./lib/app.js",
"scripts": {
"build": "tsc",
"start": "node ./lib/app.js",
"test:unit": "jest --runInBand",
"test": "npm run lint && npm run test:unit",
"watch": "concurrently --kill-others \"tsc -w\" \"nodemon --inspect ./lib/app.js\"",
"lint": "tslint -q -p . -c tslint.json",
"lint:fix": "tslint --fix -p . -c tslint.json",
"debug": "nodemon --inspect ./lib/app.js | bunyan"
},
"author": "Microsoft",
"license": "MIT",
"dependencies": {
"@microsoft/microsoft-graph-client": "^1.7.0",
"body-parser": "^1.15.2",
"bunyan": "*",
"cookie-parser": "^1.4.3",
"cookie-session": "^1.3.3",
"cookies": "^0.7.3",
"ejs": ">= 0.0.0",
"ejs-locals": ">= 0.0.0",
"express": "^4.17.1",
"express-request-id": "^1.4.1",
"method-override": "^3.0.0",
"morgan": "^1.9.1",
"node-fetch": "^2.6.0",
"passport": "*",
"passport-azure-ad": "^4.1.0",
"simple-oauth2": "^2.2.1"
},
"devDependencies": {
"@types/bunyan": "^1.8.6",
"@types/cookie-parser": "^1.4.2",
"@types/cookie-session": "^2.0.37",
"@types/cookies": "^0.7.2",
"@types/dotenv": "^6.1.1",
"@types/express": "^4.17.1",
"@types/express-request-id": "^1.4.1",
"@types/jest": "^24.0.17",
"@types/method-override": "0.0.31",
"@types/morgan": "^1.7.37",
"@types/node-fetch": "^2.5.0",
"@types/passport": "^1.0.1",
"@types/passport-azure-ad": "^4.0.3",
"@types/simple-oauth2": "^2.2.1",
"concurrently": "^4.1.1",
"dotenv": "^8.1.0",
"jest": "^24.8.0",
"nodemon": "^1.19.1",
"ts-jest": "^24.0.2",
"tslint": "^5.18.0",
"typescript": "^3.6.2"
},
"engines": {
"node": ">= 10.0.0"
}
}
+71
Ver Arquivo
@@ -0,0 +1,71 @@
<!DOCTYPE html>
<html>
<head>
<title>Simple Integration Tests</title>
</head>
<body>
<h2>Simple integration tests</h2>
<p>You must be logged in to use tests</p>
<button onclick="test_get_me()">GET /api/v1.0/me</button><br />
<button onclick="test_get_profile()">GET /api/v1.0/profile</button><br />
<button onclick="test_put_profile()">PUT /api/v1.0/profile { "foo": "bar" }</button><br />
<br />
<button onclick="test_get_connection()">GET /api/v1.0/cloudconnections/connection1</button><br />
<button onclick="test_put_connection()">PUT /api/v1.0/cloudconnections/connection1 { "foo": "bar" }</button><br />
<button onclick="test_put_connection_alt()">PUT /api/v1.0/cloudconnections/connection1 { "foo": "baz" }</button><br />
<button onclick="test_patch_connection()">PATCH /api/v1.0/cloudconnections/connection1 { "updated": ${now} }</button><br />
<button onclick="test_delete_connection()">DELETE /api/v1.0/cloudconnections/connection1</button><br />
<pre id='result'></pre>
<script>
function fetchOptions(method, thing) {
return { method, body: JSON.stringify(thing), headers: { 'Content-Type': 'application/json' } };
}
async function displayResponse(response) {
let json = null;
try { json = await response.json().catch(); } catch (err) { }
let result = (response.ok ? '*success*' : '*failed*') + '\n' + (json ? JSON.stringify(json, undefined, 2) : '');
document.getElementById("result").innerHTML = result;
}
async function test_get_me(e) {
let response = await fetch('/api/v1.0/me');
displayResponse(response);
}
async function test_get_profile(e) {
let response = await fetch('/api/v1.0/profile');
displayResponse(response);
}
async function test_put_profile(e) {
let response = await fetch('/api/v1.0/profile', fetchOptions("PUT", { foo: "bar" }));
displayResponse(response);
}
async function test_get_connection(e) {
let response = await fetch('/api/v1.0/cloudconnections/connection1');
displayResponse(response);
}
async function test_put_connection(e) {
let response = await fetch('/api/v1.0/cloudconnections/connection1', fetchOptions("PUT", { foo: "bar" }));
displayResponse(response);
}
async function test_put_connection_alt(e) {
let response = await fetch('/api/v1.0/cloudconnections/connection1', fetchOptions("PUT", { foo: "baz" }));
displayResponse(response);
}
async function test_patch_connection(e) {
let now = (new Date(Date.now())).toISOString();
let response = await fetch('/api/v1.0/cloudconnections/connection1', fetchOptions("PATCH", { updated: now }));
displayResponse(response);
}
async function test_delete_connection(e) {
let response = await fetch('/api/v1.0/cloudconnections/connection1', fetchOptions("DELETE", undefined));
displayResponse(response);
}
</script>
</body>
</html>
+8
Ver Arquivo
@@ -0,0 +1,8 @@
<% if (!user) { %>
<h2>Welcome! Please log in.</h2>
<a href="/login">Log In</a>
<% } else { %>
<p>Profile ID: <%= user.oid %></p>
<p>Email: <%= user.mail %></p>
<pre><%- JSON.stringify(user, null, 2) %></pre>
<% } %>
+14
Ver Arquivo
@@ -0,0 +1,14 @@
<% if (!user) { %>
<h2>Welcome! Please log in.</h2>
<a href="/login">Log In</a></br>
<a href="https://myapps.microsoft.com">Manage your permissions (organization)</a></br>
<a href="https://account.live.com/consent/Manage">Manage account permissions (personal)</a>
<% } else { %>
<h2>Hello, <%= user.displayName %></h2>
<a href="/account">Account Info</a></br>
<a href="/public/test.html">Run tests</a></br>
<a href="/endsession">End session</a></br>
<a href="/logout">Log Out</a></br>
<a href="https://myapps.microsoft.com">Manage your permissions (organization)</a></br>
<a href="https://account.live.com/consent/Manage">Manage account permissions (personal)</a>
<% } %>
+21
Ver Arquivo
@@ -0,0 +1,21 @@
<!DOCTYPE html>
<html>
<head>
<title>Passport-OpenID Example</title>
</head>
<body>
<% if (!user) { %>
<p>
<a href="/">Home</a> |
<a href="/login">Log In</a>
</p>
<% } else { %>
<p>
<a href="/">Home</a> |
<a href="/account">Account</a> |
<a href="/logout">Log Out</a>
</p>
<% } %>
<%- body %>
</body>
</html>
+50
Ver Arquivo
@@ -0,0 +1,50 @@
// import { default as fetch } from 'node-fetch';
import * as config from '../config';
import { app, server } from '../app';
import { ServerResponse } from 'http';
beforeAll(async (done) => {
if (!server.listening) {
server.on('listening', done())
} else {
done();
}
});
afterAll(async (done) => {
server.close(() => {
console.log('done');
done();
});
});
describe('App Server', () => {
afterAll(async (done) => {
server.close(() => {
console.log('done');
done();
});
});
test('initialized', async (done) => {
expect(app.name).toBeDefined();
expect(server.listening).toBe(true);
done();
});
test('loads app.html', async (done) => {
const response = await fetch(config.baseUrl);
expect(response.status).toBe(200);
done();
});
test('redirects to login', async (done) => {
const response = await fetch(config.baseUrl + '/api/v1.0/me');
expect(response.status).toBe(404);
done();
});
});
+13
Ver Arquivo
@@ -0,0 +1,13 @@
// import { default as fetch } from 'node-fetch';
import * as config from '../config';
describe('App Server', () => {
test('should be closed', async (done) => {
// do nothing
done();
});
});
+364
Ver Arquivo
@@ -0,0 +1,364 @@
import * as bodyParser from 'body-parser';
import * as morgan from 'morgan';
import * as bunyan from 'bunyan';
import * as cookieParser from 'cookie-parser';
import * as express from 'express';
import cookieSession = require('cookie-session');
import * as methodOverride from 'method-override';
import * as passport from 'passport';
import * as passportAzureAD from 'passport-azure-ad';
import * as config from './config';
import * as path from 'path';
import * as express_request_id from 'express-request-id';
import * as simple_oath2 from 'simple-oauth2';
import * as graph from './graph';
import * as oauth2 from './oauth2';
export const log = bunyan.createLogger({
name: 'BUNYAN-LOGGER',
src: true,
});
// -----------------------------------------------------------------------------
// Passport Setup Follows
// -----------------------------------------------------------------------------
// Setup schemas for Passport's User object and the the session persistence format.
// Augument Passport's request.user with the Azure AD oauthToken
declare global {
namespace Express {
interface User {
oid: string;
oauthToken: oauth2.Token;
}
}
}
type FullUserSchema = Express.User; // This now includes the above declaration for User.
interface MinimizedUserSchema {
oid: string;
refresh_token: string;
}
passport.serializeUser((user: FullUserSchema, done) => {
const stored: MinimizedUserSchema = { oid: user.oid, refresh_token: user.oauthToken.refresh_token };
done(null, stored);
});
passport.deserializeUser(async (stored: MinimizedUserSchema, done) => {
if (!stored || !stored.refresh_token) { return done(Error('no user profile')); }
let oauthClient = await oauth2.client(stored);
if (oauthClient.expired()) {
oauthClient = await oauthClient.refresh(); // must reassign here.
}
const profile = await graph.user(oauthClient.token.access_token).catch(reason => { log.error('could not retrieve profile', reason); });
if (!profile) { return done(Error('no user profile')); }
const result = { ...profile, oid: stored.oid, oauthToken: oauthClient.token };
return done(null, result);
});
// -----------------------------------------------------------------------------
// Define the AzureAD OIDCStrategy Strategy
// -----------------------------------------------------------------------------
const OIDCStrategyTemplate = {} as passportAzureAD.IOIDCStrategyOptionWithoutRequest;
const azureStrategyOptions: passportAzureAD.IOIDCStrategyOptionWithRequest = {
identityMetadata: config.creds.identityMetadata,
clientID: config.creds.clientID,
responseType: config.creds.responseType as typeof OIDCStrategyTemplate.responseType,
responseMode: config.creds.responseMode as typeof OIDCStrategyTemplate.responseMode,
redirectUrl: config.creds.redirectUrl,
allowHttpForRedirectUrl: config.creds.allowHttpForRedirectUrl,
clientSecret: config.creds.clientSecret,
validateIssuer: config.creds.validateIssuer,
isB2C: config.creds.isB2C,
issuer: config.creds.issuer,
passReqToCallback: true,
scope: config.creds.scope,
loggingLevel: config.creds.logLevel as typeof OIDCStrategyTemplate.loggingLevel,
nonceLifetime: config.creds.nonceLifetime,
nonceMaxAmount: config.creds.nonceMaxAmount,
useCookieInsteadOfSession: config.creds.useCookieInsteadOfSession,
cookieEncryptionKeys: config.creds.cookieEncryptionKeys,
clockSkew: config.creds.clockSkew,
};
// -----------------------------------------------------------------------------
// Use the Azure OIDCStrategy within Passport.
//
// Strategies in passport require a `verify` function, which accepts credentials
// (in this case, the `oid` claim in id_token), and invoke a callback to find
// the corresponding user object.
//
// The following are the accepted prototypes for the `verify` function
// (1) function(iss, sub, done)
// (2) function(iss, sub, profile, done)
// (3) function(iss, sub, profile, access_token, refresh_token, done)
// (4) function(iss, sub, profile, access_token, refresh_token, params, done)
// (5) function(iss, sub, profile, jwtClaims, access_token, refresh_token, params, done)
// (6) prototype (1)-(5) with an additional `req` parameter as the first parameter
//
// To do prototype (6), passReqToCallback must be set to true in the config.
// -----------------------------------------------------------------------------
passport.use(new passportAzureAD.OIDCStrategy(azureStrategyOptions, processAzureStrategy));
async function processAzureStrategy(req: express.Request,
iss: string, sub: string, profile: passportAzureAD.IProfile, jwtClaims: any,
access_token: string, refresh_token: string, oauthToken: any,
done: passportAzureAD.VerifyCallback) {
if (!profile.oid) {
return done(new Error('No oid found'), null);
}
// asynchronous verification, for effect...
process.nextTick(async () => {
const fullProfile = await graph.user(access_token);
if (!fullProfile) {
return done(Error('no profile'));
}
const oauth = oauth2.client(oauthToken);
const result = { ...fullProfile, oid: profile.oid, oauthToken: oauth.token };
return done(null, result);
});
}
// -----------------------------------------------------------------------------
// Config the express app and all the required middleware
// -----------------------------------------------------------------------------
export const app = express();
app.use(morgan(config.httpLogFormat));
app.set('trust proxy', true);
app.set('views', path.join(__dirname, '../public/views'));
app.set('view engine', 'ejs');
app.use(express_request_id());
app.use(methodOverride());
app.use(cookieParser());
app.use(cookieSession({ keys: config.creds.cookieEncryptionKeys.map(value => value.key), secure: false, maxAge: 1000 * 60 * 60 * 24 * 365 }));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(passport.initialize());
app.use(passport.session());
// -----------------------------------------------------------------------------
// Define a couple of authencation request handlers.
// -----------------------------------------------------------------------------
function ensureAuthenticated(req: express.Request, res: express.Response, next: express.NextFunction) {
if (req.isAuthenticated()) { return next(); }
res.redirect('/login');
}
function ensureAuthenticatedApi(req: express.Request, res: express.Response, next: express.NextFunction) {
if (req.isAuthenticated()) { return next(); }
res.sendStatus(401).end();
return next();
}
app.get('/', (req, res) => {
const user = { ...req.user, oauthToken: '[removed]'};
res.render('index', { user });
});
app.get('/account', ensureAuthenticated, (req, res, next) => {
const user = { ...req.user, oauthToken: '[removed]'};
res.render('account', { user });
});
app.get('/login', (req, res, next) => {
passport.authenticate('azuread-openidconnect',
{
response: res, // required
customState: 'my_state', // optional. Provide a value if you want to provide custom state value.
failureRedirect: '/',
} as passport.AuthenticateOptions,
)(req, res, next);
},
(req, res) => {
log.info('login was called');
res.redirect('/');
});
// 'GET returnURL'
// `passport.authenticate` will try to authenticate the content returned in
// query (such as authorization code). If authentication fails, user will be
// redirected to '/' (home page); otherwise, it passes to the next middleware.
app.get('/auth/openid/return', (req, res, next) => {
passport.authenticate('azuread-openidconnect',
{
response: res, // required
failureRedirect: '/',
} as passport.AuthenticateOptions,
)(req, res, next);
},
(req, res, next) => {
log.info('received a return from AzureAD.');
res.redirect('/');
});
// 'POST returnURL'
// `passport.authenticate` will try to authenticate the content returned in
// body (such as authorization code). If authentication fails, user will be
// redirected to '/' (home page); otherwise, it passes to the next middleware.
app.post('/auth/openid/return',
(req, res, next) => {
passport.authenticate('azuread-openidconnect',
{
response: res, // required
failureRedirect: '/',
} as passport.AuthenticateOptions,
)(req, res, next);
},
(req, res, next) => {
log.info('received a return from AzureAD.');
res.redirect('/');
});
// 'endsession' route, logout from passport, and destroy the session with AAD.
app.get('/endsession', (req, res) => {
req.session = null;
req.logOut();
res.redirect('/');
});
// 'logout' route, logout from passport, and destroy the session with AAD.
app.get('/logout', (req, res) => {
req.session = null;
// req.session.destroy((err) => {
req.logOut();
res.redirect(config.destroySessionUrl);
});
app.get('/api/v1.0/me', ensureAuthenticatedApi,
async (req, res, next) => {
try {
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const result = await graph.user(oauth.token.access_token);
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.get('/api/v1.0/profile', ensureAuthenticatedApi,
async (req, res, next) => {
try {
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const result = await graph.client(oauth.token.access_token).api('/me/extensions/com.code-with.vott').get();
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.put('/api/v1.0/profile', ensureAuthenticatedApi,
async (req, res, next) => {
try {
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const body = { ...req.body, extensionName: 'com.code-with.vott' };
let result = null;
try { // Handle bad graph open extension semantics
result = await graph.client(oauth.token.access_token).api('/me/extensions/').post(body);
} catch (error) {
// if it already exists and we are replacing. Delete and try again.
result = await graph.client(oauth.token.access_token).api('/me/extensions/com.code-with.vott').delete();
result = await graph.client(oauth.token.access_token).api('/me/extensions').post(body);
}
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.get('/api/v1.0/cloudconnections/:id', ensureAuthenticatedApi,
async (req, res, next) => {
try {
const id = req.params.id; // careful should only be a domain name pattern.
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const result = await graph.client(oauth.token.access_token).api(`/me/extensions/com.code-with.vott.${id}`).get();
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.put('/api/v1.0/cloudconnections/:id', ensureAuthenticatedApi,
async (req, res, next) => {
try {
const id = req.params.id; // careful should only be a domain name pattern.
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const body = { ...req.body, extensionName: `com.code-with.vott.${id}` };
let result = null;
try { // Handle bad graph open extension semantics
result = await graph.client(oauth.token.access_token).api('/me/extensions/').post(body);
} catch (error) {
// if it already exists and we are replacing. Delete and try again.
result = await graph.client(oauth.token.access_token).api(`/me/extensions/com.code-with.vott.${id}`).delete();
result = await graph.client(oauth.token.access_token).api('/me/extensions').post(body);
}
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.patch('/api/v1.0/cloudconnections/:id', ensureAuthenticatedApi,
async (req, res, next) => {
try {
const id = req.params.id; // careful should only be a domain name pattern.
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const body = { ...req.body, extensionName: `com.code-with.vott.${id}` };
const result = await graph.client(oauth.token.access_token).api(`/me/extensions/com.code-with.vott.${id}`).patch(body);
res.json(result);
res.end();
next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.delete('/api/v1.0/cloudconnections/:id', ensureAuthenticatedApi,
async (req, res, next) => {
try {
const id = req.params.id; // careful should only be a domain name pattern.
let oauth = oauth2.client(req.user.oauthToken);
if (oauth.expired()) { oauth = await oauth.refresh(); }
const result = await graph.client(oauth.token.access_token).api(`/me/extensions/com.code-with.vott.${id}`).delete();
res.end();
return next();
} catch (error) {
res.status(error.statusCode).json(error).end();
return next();
}
});
app.use('/public', express.static(path.join(__dirname, '../public')));
export const server = app.listen(config.port);
+94
Ver Arquivo
@@ -0,0 +1,94 @@
// tslint:disable-next-line: no-var-requires
require('dotenv').config();
export const baseUrl = process.env.BASE_URL || 'http://localhost:3000/';
export const redirectPath = 'auth/openid/return';
export const port = process.env.PORT || '3000';
export const loggingLevel = process.env.LOGGING_LEVEL || 'info';
export const httpLogFormat = process.env.HTTP_LOG_FORMAT || 'dev';
console.log('config values', process.env.APP_ID, process.env.APP_SECRET, baseUrl, redirectPath, port, loggingLevel, httpLogFormat);
export const creds = {
// Required
identityMetadata: 'https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration',
// 'https://login.microsoftonline.com/<tenant_name>.onmicrosoft.com/v2.0/.well-known/openid-configuration',
// or equivalently: 'https://login.microsoftonline.com/<tenant_guid>/v2.0/.well-known/openid-configuration'
//
// or you can use the common endpoint
// 'https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration'
// To use the common endpoint, you have to either turn `validateIssuer` off, or provide the `issuer` value.
// Required, the client ID of your app in AAD
clientID: process.env.APP_ID,
// Required, must be 'code', 'code id_token', 'id_token code' or 'id_token'
// If you want to get access_token, you must use 'code', 'code id_token' or 'id_token code'
responseType: 'code id_token',
// Required
responseMode: 'form_post',
// Required, the reply URL registered in AAD for your app
redirectUrl: baseUrl + redirectPath,
// Required if we use http for redirectUrl
allowHttpForRedirectUrl: process.env.ALLOW_HTTP ? process.env.ALLOW_HTTP === 'true' : true,
// Required if `responseType` is 'code', 'id_token code' or 'code id_token'.
// If app key contains '\', replace it with '\\'.
clientSecret: process.env.APP_SECRET,
// Required to set to false if you don't want to validate issuer
validateIssuer: false,
// Required if you want to provide the issuer(s) you want to validate instead of using the issuer from metadata
// issuer could be a string or an array of strings of the following form: 'https://sts.windows.net/<tenant_guid>/v2.0'
issuer: null as string,
// !Bug - must be false in this sample
// Required to set to true if the `verify` function has 'req' as the first parameter
passReqToCallback: true,
// Recommended to set to true. By default we save state in express session, if this option is set to true, then
// we encrypt state and save it in cookie instead. This option together with { session: false } allows your app
// to be completely express session free.
useCookieInsteadOfSession: true,
logLevel: loggingLevel,
// Required if `useCookieInsteadOfSession` is set to true. You can provide multiple set of key/iv pairs for key
// rollover purpose. We always use the first set of key/iv pair to encrypt cookie, but we will try every set of
// key/iv pair to decrypt cookie. Key can be any string of length 32, and iv can be any string of length 12.
cookieEncryptionKeys: (process.env.COOKIES_SECRETS ? JSON.parse(process.env.COOKIES_SECRETS) :
[
{ key: '12345678901234567890123456789012', iv: '123456789012' },
{ key: 'abcdefghijklmnopqrstuvwxyzabcdef', iv: 'abcdefghijkl' },
]) as Array<{ key: string; iv: string; }>,
// The additional scopes we want besides 'openid'.
// 'profile' scope is required, the rest scopes are optional.
// (1) if you want to receive refresh_token, use 'offline_access' scope
// (2) if you want to get access_token for graph api, use the graph api url like 'https://graph.microsoft.com/mail.read'
scope: ['profile', /* 'offline_access', */ 'https://graph.microsoft.com/user.readwrite'],
// Optional. The lifetime of nonce in session or cookie, the default value is 3600 (seconds).
nonceLifetime: null as number,
// Optional. The max amount of nonce saved in session or cookie, the default value is 10.
nonceMaxAmount: 5,
// Optional. The clock skew allowed in token validation, the default value is 300 seconds.
clockSkew: null as number,
// Optional. Is B2C
isB2C: false,
};
// The url you need to go to destroy the session with AAD
export let destroySessionUrl = 'https://login.microsoftonline.com/common/oauth2/logout?post_logout_redirect_uri=http://localhost:3000';
if (!creds.clientID || !creds.clientSecret) {
console.log('issue with config');
throw Error('Missing configuration. You need a .env file or environment variables for APP_ID and APP_SECRET');
}
+30
Ver Arquivo
@@ -0,0 +1,30 @@
import * as graphClient from '@microsoft/microsoft-graph-client';
export async function user(access_token: string) {
const token = client(access_token);
const result = await token.api('/me').get();
return result;
}
export async function getEvents(access_token: string) {
const token = client(access_token);
const events = await token.api('/me/events')
.select('subject,organizer,start,end')
.orderby('createdDateTime DESC')
.get();
return events;
}
export function client(access_token: string): graphClient.Client {
// Initialize Graph client
const result = graphClient.Client.init({
// Use the provided access token to authenticate
// requests
authProvider: (done: (err: any, access_token: string) => void) => {
done(null, access_token);
},
});
return result;
}
+27
Ver Arquivo
@@ -0,0 +1,27 @@
import * as express from 'express';
import * as simple_oauth2 from 'simple-oauth2';
import * as config from './config';
export const oauth2 = simple_oauth2.create({
client: {
id: config.creds.clientID,
secret: config.creds.clientSecret,
},
auth: {
tokenHost: 'https://login.microsoftonline.com/common',
authorizePath: '/oauth2/v2.0/authorize',
tokenPath: '/oauth2/v2.0/token',
},
});
export interface Token {
refresh_token: string;
access_token?: string;
expires_at?: string | Date;
}
export function client(token: Token) {
token.expires_at = token.expires_at || new Date(0);
const result = oauth2.accessToken.create(token);
return result;
}
+62
Ver Arquivo
@@ -0,0 +1,62 @@
{
"compilerOptions": {
/* Basic Options */
"target": "ES2018", /* Specify ECMAScript target version: 'ES3' (default), 'ES5', 'ES2015', 'ES2016', 'ES2017','ES2018' or 'ESNEXT'. */
"module": "commonjs", /* Specify module code generation: 'none', 'commonjs', 'amd', 'system', 'umd', 'es2015', or 'ESNext'. */
// "lib": [], /* Specify library files to be included in the compilation. */
"allowJs": true, /* Allow javascript files to be compiled. */
// "checkJs": true, /* Report errors in .js files. */
// "jsx": "preserve", /* Specify JSX code generation: 'preserve', 'react-native', or 'react'. */
// "declaration": true, /* Generates corresponding '.d.ts' file. */
"sourceMap": true, /* Generates corresponding '.map' file. */
// "outFile": "./", /* Concatenate and emit output to single file. */
"outDir": "./lib", /* Redirect output structure to the directory. */
"rootDir": "./src", /* Specify the root directory of input files. Use to control the output directory structure with --outDir. */
// "removeComments": true, /* Do not emit comments to output. */
// "noEmit": true, /* Do not emit outputs. */
// "importHelpers": true, /* Import emit helpers from 'tslib'. */
// "downlevelIteration": true, /* Provide full support for iterables in 'for-of', spread, and destructuring when targeting 'ES5' or 'ES3'. */
// "isolatedModules": true, /* Transpile each file as a separate module (similar to 'ts.transpileModule'). */
/* Strict Type-Checking Options */
// "strict": true, /* Enable all strict type-checking options. */
"noImplicitAny": true, /* Raise error on expressions and declarations with an implied 'any' type. */
// "strictNullChecks": true, /* Enable strict null checks. */
// "strictFunctionTypes": true, /* Enable strict checking of function types. */
// "strictPropertyInitialization": true, /* Enable strict checking of property initialization in classes. */
// "noImplicitThis": true, /* Raise error on 'this' expressions with an implied 'any' type. */
// "alwaysStrict": true, /* Parse in strict mode and emit "use strict" for each source file. */
/* Additional Checks */
// "noUnusedLocals": true, /* Report errors on unused locals. */
// "noUnusedParameters": true, /* Report errors on unused parameters. */
// "noImplicitReturns": true, /* Report error when not all code paths in function return a value. */
// "noFallthroughCasesInSwitch": true, /* Report errors for fallthrough cases in switch statement. */
/* Module Resolution Options */
"moduleResolution": "node", /* Specify module resolution strategy: 'node' (Node.js) or 'classic' (TypeScript pre-1.6). */
// "baseUrl": "./", /* Base directory to resolve non-absolute module names. */
// "paths": {}, /* A series of entries which re-map imports to lookup locations relative to the 'baseUrl'. */
// "rootDirs": [], /* List of root folders whose combined content represents the structure of the project at runtime. */
// "typeRoots": [], /* List of folders to include type definitions from. */
// "types": [], /* Type declaration files to be included in compilation. */
// "allowSyntheticDefaultImports": true, /* Allow default imports from modules with no default export. This does not affect code emit, just typechecking. */
// "esModuleInterop": true /* Enables emit interoperability between CommonJS and ES Modules via creation of namespace objects for all imports. Implies 'allowSyntheticDefaultImports'. */
// "preserveSymlinks": true, /* Do not resolve the real path of symlinks. */
/* Source Map Options */
// "sourceRoot": "./", /* Specify the location where debugger should locate TypeScript files instead of source locations. */
// "mapRoot": "./", /* Specify the location where debugger should locate map files instead of generated locations. */
// "inlineSourceMap": true, /* Emit a single file with source maps instead of having a separate file. */
// "inlineSources": true, /* Emit the source alongside the sourcemaps within a single file; requires '--inlineSourceMap' or '--sourceMap' to be set. */
/* Experimental Options */
// "experimentalDecorators": true, /* Enables experimental support for ES7 decorators. */
// "emitDecoratorMetadata": true, /* Enables experimental support for emitting type metadata for decorators. */
},
"exclude": [
"node_modules",
"coverage",
"data_models",
"public",
"lib",
"temp",
"jest.config.js",
"src/__tests__"
]
}
+48
Ver Arquivo
@@ -0,0 +1,48 @@
{
"defaultSeverity": "error",
"extends": [
"tslint:recommended"
],
"linterOptions": {
"exclude": [
"lib",
"public",
"src/routes",
"jest.config.js"
]
},
"jsRules": {},
"rules": {
"no-console": false,
"arrow-parens": false,
"max-classes-per-file": false,
"ordered-imports": false,
"object-literal-sort-keys": false,
"align": false,
"interface-name": false,
"quotemark": [
true,
"single",
"avoid-escape",
"avoid-template"
],
"max-line-length": {
"severity": "warning",
"options": [
160,
{
"ignore-pattern": "^import |^export {(.*?)} | //"
}
]
},
"variable-name": {
"options": [
"ban-keywords",
"check-format",
"allow-leading-underscore",
"allow-pascal-case",
"allow-snake-case"
]
}
}
}
+10
Ver Arquivo
@@ -12,6 +12,16 @@ import { ErrorHandler } from "./react/components/common/errorHandler/errorHandle
describe("App Component", () => {
const defaultState: IApplicationState = initialState;
const store = createReduxStore(defaultState);
const electronMock = {
ipcRenderer: {
send: jest.fn(),
on: jest.fn(),
},
};
beforeAll(() => {
delete (window as any).require;
});
function createComponent() {
return mount(
+31 -3
Ver Arquivo
@@ -115,9 +115,37 @@ describe("CNTK Export Provider", () => {
const assetsToExport = await getAssetsSpy.mock.results[0].value;
const testSplit = (100 - (defaultOptions.testTrainSplit || 80)) / 100;
const testCount = Math.ceil(assetsToExport.length * testSplit);
const testArray = assetsToExport.slice(0, testCount);
const trainArray = assetsToExport.slice(testCount, assetsToExport.length);
const trainArray = [];
const testArray = [];
const tagsAssestList: {
[index: string]: {
assetSet: Set<string>,
testArray: string[],
trainArray: string[],
},
} = {};
testProject.tags.forEach((tag) =>
tagsAssestList[tag.name] = {
assetSet: new Set(), testArray: [],
trainArray: [],
});
assetsToExport.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssestList[tagName]) {
tagsAssestList[tagName].assetSet.add(assetMetadata.asset.name);
}
});
});
});
for (const tagKey of Object.keys(tagsAssestList)) {
const assetSet = tagsAssestList[tagKey].assetSet;
const testCount = Math.ceil(assetSet.size * testSplit);
testArray.push(...Array.from(assetSet).slice(0, testCount));
trainArray.push(...Array.from(assetSet).slice(testCount, assetSet.size));
}
const storageProviderMock = LocalFileSystemProxy as any;
const writeBinaryCalls = storageProviderMock.mock.instances[0].writeBinary.mock.calls;
+8 -3
Ver Arquivo
@@ -3,6 +3,7 @@ import { ExportProvider, IExportResults } from "./exportProvider";
import { IAssetMetadata, IExportProviderOptions, IProject } from "../../models/applicationState";
import HtmlFileReader from "../../common/htmlFileReader";
import Guard from "../../common/guard";
import { splitTestAsset } from "./testAssetsSplitHelper";
enum ExportSplit {
Test,
@@ -33,13 +34,17 @@ export class CntkExportProvider extends ExportProvider<ICntkExportProviderOption
public async export(): Promise<IExportResults> {
await this.createFolderStructure();
const assetsToExport = await this.getAssetsForExport();
const testAssets: string[] = [];
const testSplit = (100 - (this.options.testTrainSplit || 80)) / 100;
const testCount = Math.ceil(assetsToExport.length * testSplit);
const testArray = assetsToExport.slice(0, testCount);
if (testSplit > 0 && testSplit <= 1) {
const splittedAssets = splitTestAsset(assetsToExport, this.project.tags, testSplit);
testAssets.push(...splittedAssets);
}
const results = await assetsToExport.mapAsync(async (assetMetadata) => {
try {
const exportSplit = testArray.find((am) => am.asset.id === assetMetadata.asset.id)
const exportSplit = testAssets.find((am) => am === assetMetadata.asset.id)
? ExportSplit.Test
: ExportSplit.Train;
+54 -9
Ver Arquivo
@@ -69,7 +69,9 @@ describe("PascalVOC Json Export Provider", () => {
beforeEach(() => {
const assetServiceMock = AssetService as jest.Mocked<typeof AssetService>;
assetServiceMock.prototype.getAssetMetadata = jest.fn((asset) => {
const mockTag = MockFactory.createTestTag();
const mockTag1 = MockFactory.createTestTag("1");
const mockTag2 = MockFactory.createTestTag("2");
const mockTag = Number(asset.id.split("-")[1]) > 7 ? mockTag1 : mockTag2;
const mockRegion1 = MockFactory.createTestRegion("region-1", [mockTag.name]);
const mockRegion2 = MockFactory.createTestRegion("region-2", [mockTag.name]);
@@ -352,27 +354,70 @@ describe("PascalVOC Json Export Provider", () => {
};
const testProject = { ...baseTestProject };
const testAssets = MockFactory.createTestAssets(10, 0);
const testAssets = MockFactory.createTestAssets(13, 0);
testAssets.forEach((asset) => asset.state = AssetState.Tagged);
testProject.assets = _.keyBy(testAssets, (asset) => asset.id);
testProject.tags = [MockFactory.createTestTag("1")];
testProject.tags = MockFactory.createTestTags(3);
const exportProvider = new PascalVOCExportProvider(testProject, options);
const getAssetsSpy = jest.spyOn(exportProvider, "getAssetsForExport");
await exportProvider.export();
const storageProviderMock = LocalFileSystemProxy as any;
const writeTextFileCalls = storageProviderMock.mock.instances[0].writeText.mock.calls as any[];
const valDataIndex = writeTextFileCalls
const valDataIndex1 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 1_val.txt"));
const trainDataIndex = writeTextFileCalls
const trainDataIndex1 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 1_train.txt"));
const valDataIndex2 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 2_val.txt"));
const trainDataIndex2 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 2_train.txt"));
const expectedTrainCount = (testTrainSplit / 100) * testAssets.length;
const expectedTestCount = ((100 - testTrainSplit) / 100) * testAssets.length;
const assetsToExport = await getAssetsSpy.mock.results[0].value;
const trainArray = [];
const testArray = [];
const tagsAssestList: {
[index: string]: {
assetSet: Set<string>,
testArray: string[],
trainArray: string[],
},
} = {};
testProject.tags.forEach((tag) =>
tagsAssestList[tag.name] = {
assetSet: new Set(), testArray: [],
trainArray: [],
});
assetsToExport.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssestList[tagName]) {
tagsAssestList[tagName].assetSet.add(assetMetadata.asset.name);
}
});
});
});
expect(writeTextFileCalls[valDataIndex][1].split("\n")).toHaveLength(expectedTestCount);
expect(writeTextFileCalls[trainDataIndex][1].split("\n")).toHaveLength(expectedTrainCount);
for (const tagKey of Object.keys(tagsAssestList)) {
const assetSet = tagsAssestList[tagKey].assetSet;
const testCount = Math.ceil(((100 - testTrainSplit) / 100) * assetSet.size);
tagsAssestList[tagKey].testArray = Array.from(assetSet).slice(0, testCount);
tagsAssestList[tagKey].trainArray = Array.from(assetSet).slice(testCount, assetSet.size);
testArray.push(...tagsAssestList[tagKey].testArray);
trainArray.push(...tagsAssestList[tagKey].trainArray);
}
expect(writeTextFileCalls[valDataIndex1][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 1"].testArray.length);
expect(writeTextFileCalls[trainDataIndex1][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 1"].trainArray.length);
expect(writeTextFileCalls[valDataIndex2][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 2"].testArray.length);
expect(writeTextFileCalls[trainDataIndex2][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 2"].trainArray.length);
}
it("Correctly generated files based on 50/50 test / train split", async () => {
+43 -24
Ver Arquivo
@@ -6,6 +6,7 @@ import HtmlFileReader from "../../common/htmlFileReader";
import { itemTemplate, annotationTemplate, objectTemplate } from "./pascalVOC/pascalVOCTemplates";
import { interpolate } from "../../common/strings";
import os from "os";
import { splitTestAsset } from "./testAssetsSplitHelper";
interface IObjectInfo {
name: string;
@@ -253,40 +254,58 @@ export class PascalVOCExportProvider extends ExportProvider<IPascalVOCExportProv
}
});
// Save ImageSets
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
if (testSplit > 0 && testSplit <= 1) {
const tags = this.project.tags;
const testAssets: string[] = splitTestAsset(allAssets, tags, testSplit);
const assetList = [];
assetUsage.forEach((tags, assetName) => {
if (tags.has(tag.name)) {
assetList.push(`${assetName} 1`);
} else {
assetList.push(`${assetName} -1`);
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
});
if (testSplit > 0 && testSplit <= 1) {
// Split in Test and Train sets
const totalAssets = assetUsage.size;
const testCount = Math.ceil(totalAssets * testSplit);
const testArray = assetList.slice(0, testCount);
const trainArray = assetList.slice(testCount, totalAssets);
const testArray = [];
const trainArray = [];
assetUsage.forEach((tags, assetName) => {
let assetString = "";
if (tags.has(tag.name)) {
assetString = `${assetName} 1`;
} else {
assetString = `${assetName} -1`;
}
if (testAssets.find((am) => am === assetName)) {
testArray.push(assetString);
} else {
trainArray.push(assetString);
}
});
const testImageSetFileName = `${imageSetsMainFolderName}/${tag.name}_val.txt`;
await this.storageProvider.writeText(testImageSetFileName, testArray.join(os.EOL));
const trainImageSetFileName = `${imageSetsMainFolderName}/${tag.name}_train.txt`;
await this.storageProvider.writeText(trainImageSetFileName, trainArray.join(os.EOL));
});
} else {
// Save ImageSets
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
const assetList = [];
assetUsage.forEach((tags, assetName) => {
if (tags.has(tag.name)) {
assetList.push(`${assetName} 1`);
} else {
assetList.push(`${assetName} -1`);
}
});
} else {
const imageSetFileName = `${imageSetsMainFolderName}/${tag.name}.txt`;
await this.storageProvider.writeText(imageSetFileName, assetList.join(os.EOL));
}
});
});
}
}
}
@@ -0,0 +1,61 @@
import _ from "lodash";
import {
IAssetMetadata, AssetState, IRegion,
RegionType, IPoint, IExportProviderOptions,
} from "../../models/applicationState";
import MockFactory from "../../common/mockFactory";
import { splitTestAsset } from "./testAssetsSplitHelper";
import { appInfo } from "../../common/appInfo";
describe("splitTestAsset Helper tests", () => {
describe("Test Train Splits", () => {
async function testTestTrainSplit(testTrainSplit: number): Promise<void> {
const assetArray = MockFactory.createTestAssets(13, 0);
const tags = MockFactory.createTestTags(2);
assetArray.forEach((asset) => asset.state = AssetState.Tagged);
const testSplit = (100 - testTrainSplit) / 100;
const testCount = Math.ceil(testSplit * assetArray.length);
const assetMetadatas = assetArray.map((asset, i) =>
MockFactory.createTestAssetMetadata(asset,
i < (assetArray.length - testCount) ?
[MockFactory.createTestRegion("Region" + i, [tags[0].name])] :
[MockFactory.createTestRegion("Region" + i, [tags[1].name])]));
const testAssetsNames = splitTestAsset(assetMetadatas, tags, testSplit);
const trainAssetsArray = assetMetadatas.filter((assetMetadata) =>
testAssetsNames.indexOf(assetMetadata.asset.name) < 0);
const testAssetsArray = assetMetadatas.filter((assetMetadata) =>
testAssetsNames.indexOf(assetMetadata.asset.name) >= 0);
const expectedTestCount = Math.ceil(testSplit * testCount) +
Math.ceil(testSplit * (assetArray.length - testCount));
expect(testAssetsNames).toHaveLength(expectedTestCount);
expect(trainAssetsArray.length + testAssetsArray.length).toEqual(assetMetadatas.length);
expect(testAssetsArray).toHaveLength(expectedTestCount);
expect(testAssetsArray.filter((assetMetadata) => assetMetadata.regions[0].tags[0] === tags[0].name).length)
.toBeGreaterThan(0);
expect(testAssetsArray.filter((assetMetadata) => assetMetadata.regions[0].tags[0] === tags[1].name).length)
.toBeGreaterThan(0);
}
it("Correctly generated files based on 50/50 test / train split", async () => {
await testTestTrainSplit(50);
});
it("Correctly generated files based on 60/40 test / train split", async () => {
await testTestTrainSplit(60);
});
it("Correctly generated files based on 80/20 test / train split", async () => {
await testTestTrainSplit(80);
});
it("Correctly generated files based on 90/10 test / train split", async () => {
await testTestTrainSplit(90);
});
});
});
@@ -0,0 +1,30 @@
import { IAssetMetadata, ITag } from "../../models/applicationState";
/**
* A helper function to split train and test assets
* @param template String containing variables
* @param params Params containing substitution values
*/
export function splitTestAsset(allAssets: IAssetMetadata[], tags: ITag[], testSplitRatio: number): string[] {
if (testSplitRatio <= 0 || testSplitRatio > 1) { return []; }
const testAssets: string[] = [];
const tagsAssetDict: { [index: string]: { assetList: Set<string> } } = {};
tags.forEach((tag) => tagsAssetDict[tag.name] = { assetList: new Set() });
allAssets.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssetDict[tagName]) {
tagsAssetDict[tagName].assetList.add(assetMetadata.asset.name);
}
});
});
});
for (const tagKey of Object.keys(tagsAssetDict)) {
const assetList = tagsAssetDict[tagKey].assetList;
const testCount = Math.ceil(assetList.size * testSplitRatio);
testAssets.push(...Array.from(assetList).slice(0, testCount));
}
return testAssets;
}
@@ -58,7 +58,7 @@ export class ConnectionPicker extends React.Component<IConnectionPickerProps, IC
<option
className="connection-option"
key={connection.id}
value={connection.id}>{connection.name}
value={connection.id}>{this.getConnectionText(connection)}
</option>)
}
</select>
@@ -71,6 +71,18 @@ export class ConnectionPicker extends React.Component<IConnectionPickerProps, IC
);
}
private getConnectionText = (connection: IConnection): string => {
const options = connection.providerOptions;
if (options["folderPath"]) {
return `${connection.name} (${options["folderPath"]})`;
} else if (options["accountName"]) {
return `${connection.name} (Azure:${options["accountName"]}\\${options["containerName"]})`;
} else {
return connection.name;
}
}
private onChange = (e) => {
const selectedConnection = this.props.connections
.find((connection) => connection.id === e.target.value) || {};
@@ -42,7 +42,7 @@ export default class EditorSideBar extends React.Component<IEditorSideBarProps,
public render() {
return (
<div className="editor-page-sidebar-nav">
<div className="editor-page-bottombar-nav">
<AutoSizer>
{({ height, width }) => (
<List
+9 -1
Ver Arquivo
@@ -4,4 +4,12 @@ import Adapter from 'enzyme-adapter-react-16';
configure({ adapter: new Adapter() });
// Silence console.log and console.group statements in testing
console.log = console.group = function() {};
console.log = console.group = function() {};
const electronMock = {
ipcRenderer: {
send: jest.fn(),
on: jest.fn(),
},
};
window.require = jest.fn(() => electronMock);