Comparar commits

...

38 Commits

Autor SHA1 Mensagem Data
Wallace Breza 68e9eddafb Revert horizontal asset bar 2019-08-23 16:02:53 -07:00
Elizabeth Halper 15881b7999 feat: move asset preview to horizontal (#876)
feat: change sidebar from vertical to horizontal

Moved the vertical asset previewer to be horizontal and at the bottom of the screen. This helps maximize the size of the picture being tagged when it is a landscape photo.

AB#860
2019-08-23 13:30:37 -07:00
hermanho 9d64f4aa0d fix: test asset distribution to include all tags on test/train split (#823)
* fix: test asset distribution to include all tags on test/train split

The test asset may not included all tags when export with test/train split option in current venison (2.1.0).

* Extract the same split logic into helper function

* Formatting

* Inverting if statement
2019-08-23 10:39:02 -07:00
Tanner Barlow c0201ca51a Revert "feat: move asset preview to horizontal (#870)" (#874)
This reverts commit 5e25dc5406.
2019-08-22 14:23:52 -07:00
Elizabeth Halper 5e25dc5406 feat: move asset preview to horizontal (#870)
feat: change sidebar from vertical to horizontal

Moved the vertical asset previewer to be horizontal and at the bottom of the screen. This helps maximize the size of the picture being tagged when it is a landscape photo.

AB#860
2019-08-22 10:59:24 -07:00
Tanner Barlow add4680e7b fix: Resolve UnhandledPromiseRejection in test (#859) 2019-08-22 10:43:38 -07:00
Tanner Barlow 60ebb41540 Merge pull request #856 from microsoft/master
release: Master into dev
2019-08-20 12:02:42 -07:00
Wallace Breza 745e854cc4 Release 2.1.0 (#790)
Updates package version and changelog for 2.1.0 release
2019-04-29 14:39:24 -07:00
Wallace Breza 2234c8a0cc fix: Updates backwards compat & fixes cntk export image bug (#789)
Fixes an issue where the images exported out of a video file were missing file extension for video projects.
2019-04-29 14:18:45 -07:00
Wallace Breza 4d02db4215 fix: Updates export options for pascalVOC rename (#788)
Adds a check during project load to update the export options if project was using previous pascalVOC name.
2019-04-29 14:18:45 -07:00
Lee, Jebum 90754dc74b fix: change method for alloc string to buffer (#777)
String.length is not appropriate for calculating buffer size
when non-alphabet letter is included in content.
Change the method Buffer.alloc to Buffer.from as directed by the nodejs document.
2019-04-29 14:18:45 -07:00
Jacopo Mangiavacchi f29963c89e feat: Add CSV Exporter (#757)
Adds CSV export provider
2019-04-29 14:18:45 -07:00
Tanner Barlow acbbc86151 fix: Fix display of tag color picker (#782)
Resolves issue of tag color picker not being shown on alt-click or color-click + edit button. Also adds several tests for increased test coverage of tagInput.tsx
2019-04-29 14:18:45 -07:00
Wallace Breza 921dbac155 feat: Active Learning Updates (#778)
Adds new active learning form
Moves active learning settings from project settings to here
Refactored and created activeLearningService
2019-04-29 14:18:45 -07:00
P.J. Little a2ef52c7a4 docs: updates to readme and changelog (#781)
Minor updates and corrections to the main readme and changelog.
2019-04-29 14:18:45 -07:00
Wallace Breza 25b4aa2dc8 Create CODE_OF_CONDUCT.md (#779)
Adds code of conduct
2019-04-29 14:18:45 -07:00
Wallace Breza 0429590bec doc: Add bug & feature templates (#780)
Adds bug and feature github templates
2019-04-29 14:18:45 -07:00
Wallace Breza 48805dcb85 test: Verify tag update/delete project actions 2019-04-29 14:18:45 -07:00
Wallace Breza 0b06d6ac5b test: Refactored editor page tests 2019-04-29 14:18:45 -07:00
Wallace Breza 3998b6efc8 fix: Refactored project tag/delete updates 2019-04-29 14:18:45 -07:00
Tanner Barlow 4a0dcb2905 Dummy commit to kick off build again 2019-04-29 14:18:45 -07:00
Tanner Barlow 354623ec21 Lint fixes 2019-04-29 14:18:45 -07:00
Tanner Barlow 8b34db5724 Clean up and docs 2019-04-29 14:18:45 -07:00
Tanner Barlow bbd83a4df5 Fix tag removal test for editor page 2019-04-29 14:18:45 -07:00
Tanner Barlow 5b4610b3d9 Rename tag function in editor page 2019-04-29 14:18:45 -07:00
Tanner Barlow 996a555333 Delete tag working 2019-04-29 14:18:45 -07:00
Tanner Barlow 8439574dc5 Saving assets in async foreach loop 2019-04-29 14:18:45 -07:00
Tanner Barlow 4193bc0e6a Register mixins and run async loop 2019-04-29 14:18:45 -07:00
Tanner Barlow f394ea3d10 Editor Page and canvas upates 2019-04-29 14:18:45 -07:00
Tanner Barlow 4f325dfe4b Split out project functions and asset functions into respective services 2019-04-29 14:18:45 -07:00
Tanner Barlow 2ef4e1387f Added confirm and strings for tag and rename operations 2019-04-29 14:18:45 -07:00
Tanner Barlow 1001528a16 Added tests for project service 2019-04-29 14:18:45 -07:00
Tanner Barlow 6974aef9d1 Project service functions 2019-04-29 14:18:45 -07:00
Tanner Barlow 37234ec2e9 refactor: Remove editor footer 2019-04-29 14:18:45 -07:00
Wallace Breza d6a059447d Fix: Enables selection of azure region for custom vision export (#765)
Adds ability to select azure region where your custom vision service is hosted.
Filters domain list by project type

Resolves #759, #770
2019-04-29 14:18:45 -07:00
Wallace Breza c10c971caf feat: CNTK Export Provider (#771)
Adds CNTK export provider into v2

Resolves #754
2019-04-29 14:18:45 -07:00
Wallace Breza 0fe63863b1 feat: Save partial project progress during project creation (#769)
This adds functionality to persist partial project information when creating a new project. Right now when creating a new connection inline within the create project flow and returning to the create project screen your partial project information is lost. Partial form progress is now saved into local storage and bound when returning to the form.

Resolves #758
2019-04-29 14:18:45 -07:00
Jacopo Mangiavacchi 39521f2b61 Fix ymax and rename Tensorflow nama everywhere (#763)
fix issue #760 [AB#18223]
2019-04-29 14:18:45 -07:00
13 arquivos alterados com 280 adições e 74 exclusões
+18
Ver Arquivo
@@ -2,6 +2,24 @@
<!-- cl-start -->
# [2.1.0](https://github.com/Microsoft/VoTT/compare/v2.0.0...v2.1.0) (04-29-2019)
[GitHub Release](https://github.com/Microsoft/VoTT/releases/tag/v2.1.0)
- fix: Updates backwards compat & fixes cntk export image bug (#789)
- fix: Updates export options for pascalVOC rename (#788)
- fix: change method for alloc string to buffer (#777)
- feat: Add CSV Exporter (#757)
- fix: Fix display of tag color picker (#782)
- feat: Active Learning Updates (#778)
- doc: updates to readme and changelog (#781)
- doc: Adds CODE_OF_CONDUCT.md (#779)
- doc: Add bug & feature templates (#780)
- fix: Refactored project tag/delete updates (#764)
- fix: Enables selection of azure region for custom vision export (#765)
- feat: CNTK Export Provider (#771)
- feat: Save partial project progress during project creation (#769)
- fix: Fixes ymax and rename Tensorflow nama everywhere (#763)
# [2.0.0](https://github.com/Microsoft/VoTT/compare/v2.0.0-preview.3...v2.0.0) (04-12-2019)
[GitHub Release](https://github.com/Microsoft/VoTT/releases/tag/v2.0.0)
+1
Ver Arquivo
@@ -28,6 +28,7 @@ VoTT helps facilitate an end-to-end machine learning pipeline:
<!-- toc -->
- [VoTT (Visual Object Tagging Tool)](#vott-visual-object-tagging-tool)
- [Table of Contents](#table-of-contents)
- [Getting Started](#getting-started)
+13 -32
Ver Arquivo
@@ -1,6 +1,6 @@
{
"name": "vott",
"version": "2.0.0",
"version": "2.1.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
@@ -7319,8 +7319,7 @@
"ansi-regex": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-2.1.1.tgz",
"integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8=",
"optional": true
"integrity": "sha1-w7M6te42DYbg5ijwRorn7yfWVN8="
},
"aproba": {
"version": "1.2.0",
@@ -7341,14 +7340,12 @@
"balanced-match": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.0.tgz",
"integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c=",
"optional": true
"integrity": "sha1-ibTRmasr7kneFk6gK4nORi1xt2c="
},
"brace-expansion": {
"version": "1.1.11",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
"integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
"optional": true,
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
@@ -7363,20 +7360,17 @@
"code-point-at": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/code-point-at/-/code-point-at-1.1.0.tgz",
"integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c=",
"optional": true
"integrity": "sha1-DQcLTQQ6W+ozovGkDi7bPZpMz3c="
},
"concat-map": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
"integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=",
"optional": true
"integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s="
},
"console-control-strings": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/console-control-strings/-/console-control-strings-1.1.0.tgz",
"integrity": "sha1-PXz0Rk22RG6mRL9LOVB/mFEAjo4=",
"optional": true
"integrity": "sha1-PXz0Rk22RG6mRL9LOVB/mFEAjo4="
},
"core-util-is": {
"version": "1.0.2",
@@ -7493,8 +7487,7 @@
"inherits": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz",
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=",
"optional": true
"integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4="
},
"ini": {
"version": "1.3.5",
@@ -7506,7 +7499,6 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-1.0.0.tgz",
"integrity": "sha1-754xOG8DGn8NZDr4L95QxFfvAMs=",
"optional": true,
"requires": {
"number-is-nan": "^1.0.0"
}
@@ -7521,7 +7513,6 @@
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz",
"integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==",
"optional": true,
"requires": {
"brace-expansion": "^1.1.7"
}
@@ -7529,14 +7520,12 @@
"minimist": {
"version": "0.0.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-0.0.8.tgz",
"integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0=",
"optional": true
"integrity": "sha1-hX/Kv8M5fSYluCKCYuhqp6ARsF0="
},
"minipass": {
"version": "2.2.4",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-2.2.4.tgz",
"integrity": "sha512-hzXIWWet/BzWhYs2b+u7dRHlruXhwdgvlTMDKC6Cb1U7ps6Ac6yQlR39xsbjWJE377YTCtKwIXIpJ5oP+j5y8g==",
"optional": true,
"requires": {
"safe-buffer": "^5.1.1",
"yallist": "^3.0.0"
@@ -7555,7 +7544,6 @@
"version": "0.5.1",
"resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.1.tgz",
"integrity": "sha1-MAV0OOrGz3+MR2fzhkjWaX11yQM=",
"optional": true,
"requires": {
"minimist": "0.0.8"
}
@@ -7636,8 +7624,7 @@
"number-is-nan": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/number-is-nan/-/number-is-nan-1.0.1.tgz",
"integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0=",
"optional": true
"integrity": "sha1-CXtgK1NCKlIsGvuHkDGDNpQaAR0="
},
"object-assign": {
"version": "4.1.1",
@@ -7649,7 +7636,6 @@
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha1-WDsap3WWHUsROsF9nFC6753Xa9E=",
"optional": true,
"requires": {
"wrappy": "1"
}
@@ -7735,8 +7721,7 @@
"safe-buffer": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz",
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg==",
"optional": true
"integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg=="
},
"safer-buffer": {
"version": "2.1.2",
@@ -7772,7 +7757,6 @@
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-1.0.2.tgz",
"integrity": "sha1-EYvfW4zcUaKn5w0hHgfisLmxB9M=",
"optional": true,
"requires": {
"code-point-at": "^1.0.0",
"is-fullwidth-code-point": "^1.0.0",
@@ -7792,7 +7776,6 @@
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-3.0.1.tgz",
"integrity": "sha1-ajhfuIU9lS1f8F0Oiq+UJ43GPc8=",
"optional": true,
"requires": {
"ansi-regex": "^2.0.0"
}
@@ -7836,14 +7819,12 @@
"wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=",
"optional": true
"integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8="
},
"yallist": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-3.0.2.tgz",
"integrity": "sha1-hFK0u36Dx8GI2AQcGoN8dz1ti7k=",
"optional": true
"integrity": "sha1-hFK0u36Dx8GI2AQcGoN8dz1ti7k="
}
}
},
@@ -14828,7 +14809,7 @@
"react-modal": {
"version": "3.8.1",
"resolved": "https://registry.npmjs.org/react-modal/-/react-modal-3.8.1.tgz",
"integrity": "sha1-cwD5Sm+SouF5lN4L5sy2FzRGTJ4=",
"integrity": "sha512-aLKeZM9pgXpIKVwopRHMuvqKWiBajkqisDA8UzocdCF6S4fyKVfLWmZR5G1Q0ODBxxxxf2XIwiCP8G/11GJAuw==",
"requires": {
"exenv": "^1.2.0",
"prop-types": "^15.5.10",
+1 -1
Ver Arquivo
@@ -1,6 +1,6 @@
{
"name": "vott",
"version": "2.0.0",
"version": "2.1.0",
"author": {
"name": "Microsoft",
"url": "https://github.com/Microsoft/VoTT"
+10
Ver Arquivo
@@ -12,6 +12,16 @@ import { ErrorHandler } from "./react/components/common/errorHandler/errorHandle
describe("App Component", () => {
const defaultState: IApplicationState = initialState;
const store = createReduxStore(defaultState);
const electronMock = {
ipcRenderer: {
send: jest.fn(),
on: jest.fn(),
},
};
beforeAll(() => {
delete (window as any).require;
});
function createComponent() {
return mount(
+31 -3
Ver Arquivo
@@ -115,9 +115,37 @@ describe("CNTK Export Provider", () => {
const assetsToExport = await getAssetsSpy.mock.results[0].value;
const testSplit = (100 - (defaultOptions.testTrainSplit || 80)) / 100;
const testCount = Math.ceil(assetsToExport.length * testSplit);
const testArray = assetsToExport.slice(0, testCount);
const trainArray = assetsToExport.slice(testCount, assetsToExport.length);
const trainArray = [];
const testArray = [];
const tagsAssestList: {
[index: string]: {
assetSet: Set<string>,
testArray: string[],
trainArray: string[],
},
} = {};
testProject.tags.forEach((tag) =>
tagsAssestList[tag.name] = {
assetSet: new Set(), testArray: [],
trainArray: [],
});
assetsToExport.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssestList[tagName]) {
tagsAssestList[tagName].assetSet.add(assetMetadata.asset.name);
}
});
});
});
for (const tagKey of Object.keys(tagsAssestList)) {
const assetSet = tagsAssestList[tagKey].assetSet;
const testCount = Math.ceil(assetSet.size * testSplit);
testArray.push(...Array.from(assetSet).slice(0, testCount));
trainArray.push(...Array.from(assetSet).slice(testCount, assetSet.size));
}
const storageProviderMock = LocalFileSystemProxy as any;
const writeBinaryCalls = storageProviderMock.mock.instances[0].writeBinary.mock.calls;
+8 -3
Ver Arquivo
@@ -3,6 +3,7 @@ import { ExportProvider, IExportResults } from "./exportProvider";
import { IAssetMetadata, IExportProviderOptions, IProject } from "../../models/applicationState";
import HtmlFileReader from "../../common/htmlFileReader";
import Guard from "../../common/guard";
import { splitTestAsset } from "./testAssetsSplitHelper";
enum ExportSplit {
Test,
@@ -33,13 +34,17 @@ export class CntkExportProvider extends ExportProvider<ICntkExportProviderOption
public async export(): Promise<IExportResults> {
await this.createFolderStructure();
const assetsToExport = await this.getAssetsForExport();
const testAssets: string[] = [];
const testSplit = (100 - (this.options.testTrainSplit || 80)) / 100;
const testCount = Math.ceil(assetsToExport.length * testSplit);
const testArray = assetsToExport.slice(0, testCount);
if (testSplit > 0 && testSplit <= 1) {
const splittedAssets = splitTestAsset(assetsToExport, this.project.tags, testSplit);
testAssets.push(...splittedAssets);
}
const results = await assetsToExport.mapAsync(async (assetMetadata) => {
try {
const exportSplit = testArray.find((am) => am.asset.id === assetMetadata.asset.id)
const exportSplit = testAssets.find((am) => am === assetMetadata.asset.id)
? ExportSplit.Test
: ExportSplit.Train;
+54 -9
Ver Arquivo
@@ -69,7 +69,9 @@ describe("PascalVOC Json Export Provider", () => {
beforeEach(() => {
const assetServiceMock = AssetService as jest.Mocked<typeof AssetService>;
assetServiceMock.prototype.getAssetMetadata = jest.fn((asset) => {
const mockTag = MockFactory.createTestTag();
const mockTag1 = MockFactory.createTestTag("1");
const mockTag2 = MockFactory.createTestTag("2");
const mockTag = Number(asset.id.split("-")[1]) > 7 ? mockTag1 : mockTag2;
const mockRegion1 = MockFactory.createTestRegion("region-1", [mockTag.name]);
const mockRegion2 = MockFactory.createTestRegion("region-2", [mockTag.name]);
@@ -352,27 +354,70 @@ describe("PascalVOC Json Export Provider", () => {
};
const testProject = { ...baseTestProject };
const testAssets = MockFactory.createTestAssets(10, 0);
const testAssets = MockFactory.createTestAssets(13, 0);
testAssets.forEach((asset) => asset.state = AssetState.Tagged);
testProject.assets = _.keyBy(testAssets, (asset) => asset.id);
testProject.tags = [MockFactory.createTestTag("1")];
testProject.tags = MockFactory.createTestTags(3);
const exportProvider = new PascalVOCExportProvider(testProject, options);
const getAssetsSpy = jest.spyOn(exportProvider, "getAssetsForExport");
await exportProvider.export();
const storageProviderMock = LocalFileSystemProxy as any;
const writeTextFileCalls = storageProviderMock.mock.instances[0].writeText.mock.calls as any[];
const valDataIndex = writeTextFileCalls
const valDataIndex1 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 1_val.txt"));
const trainDataIndex = writeTextFileCalls
const trainDataIndex1 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 1_train.txt"));
const valDataIndex2 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 2_val.txt"));
const trainDataIndex2 = writeTextFileCalls
.findIndex((args) => args[0].endsWith("/ImageSets/Main/Tag 2_train.txt"));
const expectedTrainCount = (testTrainSplit / 100) * testAssets.length;
const expectedTestCount = ((100 - testTrainSplit) / 100) * testAssets.length;
const assetsToExport = await getAssetsSpy.mock.results[0].value;
const trainArray = [];
const testArray = [];
const tagsAssestList: {
[index: string]: {
assetSet: Set<string>,
testArray: string[],
trainArray: string[],
},
} = {};
testProject.tags.forEach((tag) =>
tagsAssestList[tag.name] = {
assetSet: new Set(), testArray: [],
trainArray: [],
});
assetsToExport.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssestList[tagName]) {
tagsAssestList[tagName].assetSet.add(assetMetadata.asset.name);
}
});
});
});
expect(writeTextFileCalls[valDataIndex][1].split("\n")).toHaveLength(expectedTestCount);
expect(writeTextFileCalls[trainDataIndex][1].split("\n")).toHaveLength(expectedTrainCount);
for (const tagKey of Object.keys(tagsAssestList)) {
const assetSet = tagsAssestList[tagKey].assetSet;
const testCount = Math.ceil(((100 - testTrainSplit) / 100) * assetSet.size);
tagsAssestList[tagKey].testArray = Array.from(assetSet).slice(0, testCount);
tagsAssestList[tagKey].trainArray = Array.from(assetSet).slice(testCount, assetSet.size);
testArray.push(...tagsAssestList[tagKey].testArray);
trainArray.push(...tagsAssestList[tagKey].trainArray);
}
expect(writeTextFileCalls[valDataIndex1][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 1"].testArray.length);
expect(writeTextFileCalls[trainDataIndex1][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 1"].trainArray.length);
expect(writeTextFileCalls[valDataIndex2][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 2"].testArray.length);
expect(writeTextFileCalls[trainDataIndex2][1].split(/\r?\n/).filter((line) =>
line.endsWith(" 1"))).toHaveLength(tagsAssestList["Tag 2"].trainArray.length);
}
it("Correctly generated files based on 50/50 test / train split", async () => {
+43 -24
Ver Arquivo
@@ -6,6 +6,7 @@ import HtmlFileReader from "../../common/htmlFileReader";
import { itemTemplate, annotationTemplate, objectTemplate } from "./pascalVOC/pascalVOCTemplates";
import { interpolate } from "../../common/strings";
import os from "os";
import { splitTestAsset } from "./testAssetsSplitHelper";
interface IObjectInfo {
name: string;
@@ -253,40 +254,58 @@ export class PascalVOCExportProvider extends ExportProvider<IPascalVOCExportProv
}
});
// Save ImageSets
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
if (testSplit > 0 && testSplit <= 1) {
const tags = this.project.tags;
const testAssets: string[] = splitTestAsset(allAssets, tags, testSplit);
const assetList = [];
assetUsage.forEach((tags, assetName) => {
if (tags.has(tag.name)) {
assetList.push(`${assetName} 1`);
} else {
assetList.push(`${assetName} -1`);
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
});
if (testSplit > 0 && testSplit <= 1) {
// Split in Test and Train sets
const totalAssets = assetUsage.size;
const testCount = Math.ceil(totalAssets * testSplit);
const testArray = assetList.slice(0, testCount);
const trainArray = assetList.slice(testCount, totalAssets);
const testArray = [];
const trainArray = [];
assetUsage.forEach((tags, assetName) => {
let assetString = "";
if (tags.has(tag.name)) {
assetString = `${assetName} 1`;
} else {
assetString = `${assetName} -1`;
}
if (testAssets.find((am) => am === assetName)) {
testArray.push(assetString);
} else {
trainArray.push(assetString);
}
});
const testImageSetFileName = `${imageSetsMainFolderName}/${tag.name}_val.txt`;
await this.storageProvider.writeText(testImageSetFileName, testArray.join(os.EOL));
const trainImageSetFileName = `${imageSetsMainFolderName}/${tag.name}_train.txt`;
await this.storageProvider.writeText(trainImageSetFileName, trainArray.join(os.EOL));
});
} else {
// Save ImageSets
await tags.forEachAsync(async (tag) => {
const tagInstances = tagUsage.get(tag.name) || 0;
if (!exportUnassignedTags && tagInstances === 0) {
return;
}
const assetList = [];
assetUsage.forEach((tags, assetName) => {
if (tags.has(tag.name)) {
assetList.push(`${assetName} 1`);
} else {
assetList.push(`${assetName} -1`);
}
});
} else {
const imageSetFileName = `${imageSetsMainFolderName}/${tag.name}.txt`;
await this.storageProvider.writeText(imageSetFileName, assetList.join(os.EOL));
}
});
});
}
}
}
@@ -0,0 +1,61 @@
import _ from "lodash";
import {
IAssetMetadata, AssetState, IRegion,
RegionType, IPoint, IExportProviderOptions,
} from "../../models/applicationState";
import MockFactory from "../../common/mockFactory";
import { splitTestAsset } from "./testAssetsSplitHelper";
import { appInfo } from "../../common/appInfo";
describe("splitTestAsset Helper tests", () => {
describe("Test Train Splits", () => {
async function testTestTrainSplit(testTrainSplit: number): Promise<void> {
const assetArray = MockFactory.createTestAssets(13, 0);
const tags = MockFactory.createTestTags(2);
assetArray.forEach((asset) => asset.state = AssetState.Tagged);
const testSplit = (100 - testTrainSplit) / 100;
const testCount = Math.ceil(testSplit * assetArray.length);
const assetMetadatas = assetArray.map((asset, i) =>
MockFactory.createTestAssetMetadata(asset,
i < (assetArray.length - testCount) ?
[MockFactory.createTestRegion("Region" + i, [tags[0].name])] :
[MockFactory.createTestRegion("Region" + i, [tags[1].name])]));
const testAssetsNames = splitTestAsset(assetMetadatas, tags, testSplit);
const trainAssetsArray = assetMetadatas.filter((assetMetadata) =>
testAssetsNames.indexOf(assetMetadata.asset.name) < 0);
const testAssetsArray = assetMetadatas.filter((assetMetadata) =>
testAssetsNames.indexOf(assetMetadata.asset.name) >= 0);
const expectedTestCount = Math.ceil(testSplit * testCount) +
Math.ceil(testSplit * (assetArray.length - testCount));
expect(testAssetsNames).toHaveLength(expectedTestCount);
expect(trainAssetsArray.length + testAssetsArray.length).toEqual(assetMetadatas.length);
expect(testAssetsArray).toHaveLength(expectedTestCount);
expect(testAssetsArray.filter((assetMetadata) => assetMetadata.regions[0].tags[0] === tags[0].name).length)
.toBeGreaterThan(0);
expect(testAssetsArray.filter((assetMetadata) => assetMetadata.regions[0].tags[0] === tags[1].name).length)
.toBeGreaterThan(0);
}
it("Correctly generated files based on 50/50 test / train split", async () => {
await testTestTrainSplit(50);
});
it("Correctly generated files based on 60/40 test / train split", async () => {
await testTestTrainSplit(60);
});
it("Correctly generated files based on 80/20 test / train split", async () => {
await testTestTrainSplit(80);
});
it("Correctly generated files based on 90/10 test / train split", async () => {
await testTestTrainSplit(90);
});
});
});
@@ -0,0 +1,30 @@
import { IAssetMetadata, ITag } from "../../models/applicationState";
/**
* A helper function to split train and test assets
* @param template String containing variables
* @param params Params containing substitution values
*/
export function splitTestAsset(allAssets: IAssetMetadata[], tags: ITag[], testSplitRatio: number): string[] {
if (testSplitRatio <= 0 || testSplitRatio > 1) { return []; }
const testAssets: string[] = [];
const tagsAssetDict: { [index: string]: { assetList: Set<string> } } = {};
tags.forEach((tag) => tagsAssetDict[tag.name] = { assetList: new Set() });
allAssets.forEach((assetMetadata) => {
assetMetadata.regions.forEach((region) => {
region.tags.forEach((tagName) => {
if (tagsAssetDict[tagName]) {
tagsAssetDict[tagName].assetList.add(assetMetadata.asset.name);
}
});
});
});
for (const tagKey of Object.keys(tagsAssetDict)) {
const assetList = tagsAssetDict[tagKey].assetList;
const testCount = Math.ceil(assetList.size * testSplitRatio);
testAssets.push(...Array.from(assetList).slice(0, testCount));
}
return testAssets;
}
@@ -42,7 +42,7 @@ export default class EditorSideBar extends React.Component<IEditorSideBarProps,
public render() {
return (
<div className="editor-page-sidebar-nav">
<div className="editor-page-bottombar-nav">
<AutoSizer>
{({ height, width }) => (
<List
+9 -1
Ver Arquivo
@@ -4,4 +4,12 @@ import Adapter from 'enzyme-adapter-react-16';
configure({ adapter: new Adapter() });
// Silence console.log and console.group statements in testing
console.log = console.group = function() {};
console.log = console.group = function() {};
const electronMock = {
ipcRenderer: {
send: jest.fn(),
on: jest.fn(),
},
};
window.require = jest.fn(() => electronMock);