Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project export #3365

Merged
merged 68 commits into from
Aug 6, 2021
Merged
Show file tree
Hide file tree
Changes from 55 commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
c53e062
init
ActiveChooN Jun 22, 2021
46e89b0
Fixed typos
ActiveChooN Jun 23, 2021
22fafe3
Merge branch 'develop' into dk/project-export
ActiveChooN Jun 24, 2021
7f4a349
Fixed cache time value
ActiveChooN Jun 24, 2021
4a5bc3a
Added MOTS PNG
ActiveChooN Jun 28, 2021
c57ab56
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 1, 2021
04d14c5
temp
ActiveChooN Jul 2, 2021
f6b833f
Another temp
ActiveChooN Jul 7, 2021
d45bec8
WIP on UI
ActiveChooN Jul 7, 2021
b23a9ba
Added working modal
ActiveChooN Jul 8, 2021
4c22e11
Moved task export to modal
ActiveChooN Jul 9, 2021
a4d6094
Fixed vscode files
ActiveChooN Jul 9, 2021
aadc83c
Removed task actions
ActiveChooN Jul 9, 2021
bcd2034
Fixed image_maker calling
ActiveChooN Jul 12, 2021
3aa16a1
Separated DataExtractor class
ActiveChooN Jul 12, 2021
b59ab03
wip
ActiveChooN Jul 14, 2021
5d9b512
Added export with CVAT format
ActiveChooN Jul 15, 2021
a914d05
Fixed format resetting
ActiveChooN Jul 15, 2021
3b480fa
Fixed dir creating
ActiveChooN Jul 15, 2021
2647eb9
Fixed export in CVAT formats
ActiveChooN Jul 16, 2021
15d226a
Added simple server tests, fixed datumaro format export
ActiveChooN Jul 20, 2021
3da98c4
Reverted settings change
ActiveChooN Jul 20, 2021
d70ee77
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 20, 2021
b8b3d9c
Fixed merge
ActiveChooN Jul 20, 2021
e1f8db6
Fixed some comments
ActiveChooN Jul 21, 2021
98c9083
Update cvat/apps/dataset_manager/bindings.py
ActiveChooN Jul 21, 2021
84f8b7a
Fixed comment
ActiveChooN Jul 21, 2021
4f8d28d
Merge branch 'dk/project-export' of https://github.com/openvinotoolki…
ActiveChooN Jul 21, 2021
29ffac1
Added restrictions for 3d tasks
ActiveChooN Jul 22, 2021
4e2aac1
Fixed validator
ActiveChooN Jul 22, 2021
5a48dfc
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 23, 2021
ff19e71
Fixed linter and tests
ActiveChooN Jul 23, 2021
3e6d7df
Merge remote-tracking branch 'origin/develop' into dk/project-export
ActiveChooN Jul 26, 2021
dc45b9f
Fixed comments
ActiveChooN Jul 27, 2021
6c683ac
Added file extesion to the form
ActiveChooN Jul 27, 2021
abdb59f
Added notification
ActiveChooN Jul 27, 2021
f4c4e87
Fixed header
ActiveChooN Jul 27, 2021
76c073c
Added default format for tasks
ActiveChooN Jul 27, 2021
370f337
Fixed test
ActiveChooN Jul 28, 2021
141f6a6
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 29, 2021
c076584
Fixed notification
ActiveChooN Jul 29, 2021
1f5f6f2
Added some classes
dvkruchinin Jul 29, 2021
7d1b9cf
Tests adaptation
dvkruchinin Jul 29, 2021
3be269c
Merge branch 'dk/project-export' of https://github.com/openvinotoolki…
dvkruchinin Jul 29, 2021
f129f76
Rename classes
dvkruchinin Jul 29, 2021
dc4b56a
The next tests adapdation
dvkruchinin Jul 29, 2021
cd67f5b
Additional tests adaptations.
dvkruchinin Jul 29, 2021
c955995
Some rework main.yaml
dvkruchinin Jul 29, 2021
7e9fcaa
Added debug
dvkruchinin Jul 29, 2021
4be0540
Debug removed
dvkruchinin Jul 29, 2021
415e275
Fixed late binding problem
ActiveChooN Jul 30, 2021
e6c95a2
Fixed 3d default format
ActiveChooN Jul 30, 2021
4a66693
Merge branch 'develop' into dk/project-export
ActiveChooN Jul 30, 2021
c7ed863
Merge pull request #3485 from dvkruchinin/dkru/project-export-adjast-…
ActiveChooN Jul 30, 2021
0687808
Added support for project with 3d tasks
ActiveChooN Aug 2, 2021
7e412f4
Fixed test
ActiveChooN Aug 2, 2021
4fe5598
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 3, 2021
e635db3
Update cvat/apps/dataset_manager/views.py
ActiveChooN Aug 3, 2021
e9f1fc1
Fixed function name
ActiveChooN Aug 3, 2021
0a06baf
Revert "Added support for project with 3d tasks"
ActiveChooN Aug 3, 2021
1387b2f
Changed defaulted subset
ActiveChooN Aug 3, 2021
cbb7a1e
Update cvat/apps/dataset_manager/bindings.py
ActiveChooN Aug 3, 2021
566e490
Fixed project extractor
ActiveChooN Aug 3, 2021
42a974e
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 4, 2021
333bc88
Fixed project export cache invalidation
ActiveChooN Aug 5, 2021
9952d51
Merge branch 'develop' into dk/project-export
ActiveChooN Aug 5, 2021
f5c89ef
Added CHANGELOG, increased versions
ActiveChooN Aug 5, 2021
a73122f
Merge branch 'develop' into dk/project-export
Aug 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ on:
- 'master'
- 'develop'
pull_request:
branches:
- '*'
nmanovic marked this conversation as resolved.
Show resolved Hide resolved

jobs:
Unit_testing:
runs-on: ubuntu-latest
Expand Down
43 changes: 19 additions & 24 deletions cvat-core/src/annotations.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (C) 2019-2020 Intel Corporation
// Copyright (C) 2019-2021 Intel Corporation
//
// SPDX-License-Identifier: MIT

Expand All @@ -8,8 +8,9 @@
const AnnotationsSaver = require('./annotations-saver');
const AnnotationsHistory = require('./annotations-history');
const { checkObjectType } = require('./common');
const { Task } = require('./session');
const { Loader, Dumper } = require('./annotation-formats');
const { Project } = require('./project');
const { Task, Job } = require('./session');
const { Loader } = require('./annotation-formats');
const { ScriptingError, DataError, ArgumentError } = require('./exceptions');

const jobCache = new WeakMap();
Expand Down Expand Up @@ -50,6 +51,7 @@
stopFrame,
frameMeta,
});
// eslint-disable-next-line no-unsanitized/method
collection.import(rawAnnotations);

const saver = new AnnotationsSaver(rawAnnotations.version, collection, session);
Expand Down Expand Up @@ -232,27 +234,12 @@
await serverProxy.annotations.uploadAnnotations(sessionType, session.id, file, loader.name);
}

async function dumpAnnotations(session, name, dumper) {
if (!(dumper instanceof Dumper)) {
throw new ArgumentError('A dumper must be instance of Dumper class');
}

let result = null;
const sessionType = session instanceof Task ? 'task' : 'job';
if (sessionType === 'job') {
result = await serverProxy.annotations.dumpAnnotations(session.task.id, name, dumper.name);
} else {
result = await serverProxy.annotations.dumpAnnotations(session.id, name, dumper.name);
}

return result;
}

function importAnnotations(session, data) {
const sessionType = session instanceof Task ? 'task' : 'job';
const cache = getCache(sessionType);

if (cache.has(session)) {
// eslint-disable-next-line no-unsanitized/method
return cache.get(session).collection.import(data);
}

Expand All @@ -274,16 +261,25 @@
);
}

async function exportDataset(session, format) {
async function exportDataset(instance, format, name, saveImages = false) {
if (!(format instanceof String || typeof format === 'string')) {
throw new ArgumentError('Format must be a string');
}
if (!(session instanceof Task)) {
throw new ArgumentError('A dataset can only be created from a task');
if (!(instance instanceof Task || instance instanceof Project || instance instanceof Job)) {
throw new ArgumentError('A dataset can only be created from a job, task or project');
}
if (typeof saveImages !== 'boolean') {
throw new ArgumentError('Save images parameter must be a boolean');
}

let result = null;
result = await serverProxy.tasks.exportDataset(session.id, format);
if (instance instanceof Task) {
result = await serverProxy.tasks.exportDataset(instance.id, format, name, saveImages);
} else if (instance instanceof Job) {
result = await serverProxy.tasks.exportDataset(instance.task.id, format, name, saveImages);
} else {
result = await serverProxy.projects.exportDataset(instance.id, format, name, saveImages);
}

return result;
}
Expand Down Expand Up @@ -367,7 +363,6 @@
annotationsStatistics,
selectObject,
uploadAnnotations,
dumpAnnotations,
importAnnotations,
exportAnnotations,
exportDataset,
Expand Down
3 changes: 2 additions & 1 deletion cvat-core/src/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ function build() {
const Review = require('./review');
const { Job, Task } = require('./session');
const { Project } = require('./project');
const implementProject = require('./project-implementation');
const { Attribute, Label } = require('./labels');
const MLModel = require('./ml-model');
const { FrameData } = require('./frames');
Expand Down Expand Up @@ -754,7 +755,7 @@ function build() {
*/
classes: {
User,
Project,
Project: implementProject(Project),
Task,
Job,
Log,
Expand Down
74 changes: 74 additions & 0 deletions cvat-core/src/project-implementation.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
// Copyright (C) 2021 Intel Corporation
//
// SPDX-License-Identifier: MIT

(() => {
const serverProxy = require('./server-proxy');
Comment on lines +5 to +6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest passing a prototype of Project as an argument as it was done in api-implementation.

const { getPreview } = require('./frames');

const { Project } = require('./project');
const { exportDataset } = require('./annotations');

function implementProject(projectClass) {
projectClass.prototype.save.implementation = async function () {
const trainingProjectCopy = this.trainingProject;
if (typeof this.id !== 'undefined') {
// project has been already created, need to update some data
const projectData = {
name: this.name,
assignee_id: this.assignee ? this.assignee.id : null,
bug_tracker: this.bugTracker,
labels: [...this._internalData.labels.map((el) => el.toJSON())],
};

if (trainingProjectCopy) {
projectData.training_project = trainingProjectCopy;
}

await serverProxy.projects.save(this.id, projectData);
return this;
}

// initial creating
const projectSpec = {
name: this.name,
labels: [...this.labels.map((el) => el.toJSON())],
};

if (this.bugTracker) {
projectSpec.bug_tracker = this.bugTracker;
}

if (trainingProjectCopy) {
projectSpec.training_project = trainingProjectCopy;
}

const project = await serverProxy.projects.create(projectSpec);
return new Project(project);
};

projectClass.prototype.delete.implementation = async function () {
const result = await serverProxy.projects.delete(this.id);
return result;
};

projectClass.prototype.preview.implementation = async function () {
if (!this._internalData.task_ids.length) {
return '';
}
const frameData = await getPreview(this._internalData.task_ids[0]);
return frameData;
};

projectClass.prototype.annotations.exportDataset.implementation = async function (
format, saveImages, customName,
) {
const result = exportDataset(this, format, customName, saveImages);
return result;
};

return projectClass;
}

module.exports = implementProject;
})();
86 changes: 31 additions & 55 deletions cvat-core/src/project.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,9 @@

(() => {
const PluginRegistry = require('./plugins');
const serverProxy = require('./server-proxy');
const { ArgumentError } = require('./exceptions');
const { Task } = require('./session');
const { Label } = require('./labels');
const { getPreview } = require('./frames');
const User = require('./user');

/**
Expand Down Expand Up @@ -203,7 +201,7 @@
},
},
/**
* Tasks linked with the project
* Tasks related with the project
* @name tasks
* @type {module:API.cvat.classes.Task[]}
* @memberof module:API.cvat.classes.Project
Expand All @@ -214,7 +212,7 @@
get: () => [...data.tasks],
},
/**
* Subsets array for linked tasks
* Subsets array for related tasks
* @name subsets
* @type {string[]}
* @memberof module:API.cvat.classes.Project
Expand Down Expand Up @@ -254,6 +252,13 @@
},
}),
);

// When we call a function, for example: project.annotations.get()
// In the method get we lose the project context
// So, we need return it
this.annotations = {
exportDataset: Object.getPrototypeOf(this).annotations.exportDataset.bind(this),
};
}

/**
Expand Down Expand Up @@ -289,7 +294,7 @@
}

/**
* Method deletes a task from a server
* Method deletes a project from a server
* @method delete
* @memberof module:API.cvat.classes.Project
* @readonly
Expand All @@ -304,57 +309,28 @@
}
}

Object.defineProperties(
Project.prototype,
Object.freeze({
annotations: Object.freeze({
value: {
async exportDataset(format, saveImages, customName = '') {
const result = await PluginRegistry.apiWrapper.call(
this,
Project.prototype.annotations.exportDataset,
format,
saveImages,
customName,
);
return result;
},
},
writable: true,
}),
}),
);

module.exports = {
Project,
};

Project.prototype.save.implementation = async function () {
const trainingProjectCopy = this.trainingProject;
if (typeof this.id !== 'undefined') {
// project has been already created, need to update some data
const projectData = {
name: this.name,
assignee_id: this.assignee ? this.assignee.id : null,
bug_tracker: this.bugTracker,
labels: [...this._internalData.labels.map((el) => el.toJSON())],
};

if (trainingProjectCopy) {
projectData.training_project = trainingProjectCopy;
}

await serverProxy.projects.save(this.id, projectData);
return this;
}

// initial creating
const projectSpec = {
name: this.name,
labels: [...this.labels.map((el) => el.toJSON())],
};

if (this.bugTracker) {
projectSpec.bug_tracker = this.bugTracker;
}

if (trainingProjectCopy) {
projectSpec.training_project = trainingProjectCopy;
}

const project = await serverProxy.projects.create(projectSpec);
return new Project(project);
};

Project.prototype.delete.implementation = async function () {
const result = await serverProxy.projects.delete(this.id);
return result;
};

Project.prototype.preview.implementation = async function () {
if (!this._internalData.task_ids.length) {
return '';
}
const frameData = await getPreview(this._internalData.task_ids[0]);
return frameData;
};
})();
Loading