Skip to content
This repository has been archived by the owner on Nov 14, 2023. It is now read-only.

Commit

Permalink
Fixed SAM plugin (403 code for workers in organizations) (cvat-ai#6514)
Browse files Browse the repository at this point in the history
<!-- Raise an issue to propose your change
(https://github.com/opencv/cvat/issues).
It helps to avoid duplication of efforts from multiple independent
contributors.
Discuss your ideas with maintainers to be sure that changes will be
approved and merged.
Read the [Contribution
guide](https://opencv.github.io/cvat/docs/contributing/). -->

<!-- Provide a general summary of your changes in the Title above -->

### Motivation and context
Resolved cvat-ai#6221

### How has this been tested?
<!-- Please describe in detail how you tested your changes.
Include details of your testing environment, and the tests you ran to
see how your change affects other areas of the code, etc. -->

### Checklist
<!-- Go over all the following points, and put an `x` in all the boxes
that apply.
If an item isn't applicable for some reason, then ~~explicitly
strikethrough~~ the whole
line. If you don't do that, GitHub will show incorrect progress for the
pull request.
If you're unsure about any of these, don't hesitate to ask. We're here
to help! -->
- [x] I submit my changes into the `develop` branch
- [x] I have added a description of my changes into the
[CHANGELOG](https://github.com/opencv/cvat/blob/develop/CHANGELOG.md)
file
- [ ] I have updated the documentation accordingly
- [ ] I have added tests to cover my changes
- [x] I have linked related issues (see [GitHub docs](

https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword))
- [ ] I have increased versions of npm packages if it is necessary

([cvat-canvas](https://github.com/opencv/cvat/tree/develop/cvat-canvas#versioning),

[cvat-core](https://github.com/opencv/cvat/tree/develop/cvat-core#versioning),

[cvat-data](https://github.com/opencv/cvat/tree/develop/cvat-data#versioning)
and

[cvat-ui](https://github.com/opencv/cvat/tree/develop/cvat-ui#versioning))

### License

- [x] I submit _my code changes_ under the same [MIT License](
https://github.com/opencv/cvat/blob/develop/LICENSE) that covers the
project.
  Feel free to contact the maintainers if that's a concern.

---------

Co-authored-by: Kirill Sizov <kirill.sizov@cvat.ai>
  • Loading branch information
2 people authored and mikhail-treskin committed Oct 25, 2023
1 parent 4ae1862 commit 7f03eb0
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 8 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
(<https://github.com/opencv/cvat/pull/6454>)
- \[SDK\] SDK should not change input data in models (<https://github.com/opencv/cvat/pull/6455>)
- 3D job can not be opened in validation mode (<https://github.com/opencv/cvat/pull/6507>)
- Fixed SAM plugin (403 code for workers in organizations) (<https://github.com/opencv/cvat/pull/6514>)
- Memory leak related to unclosed av container (<https://github.com/opencv/cvat/pull/6501>)
- Using initial frame from query parameter to open specific frame in a job
(<https://github.com/opencv/cvat/pull/6506>)
Expand Down
43 changes: 35 additions & 8 deletions cvat-ui/plugins/sam_plugin/src/ts/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,19 @@ interface SAMPlugin {
) => Promise<any>;
};
};
jobs: {
get: {
leave: (
plugin: SAMPlugin,
results: any[],
query: { jobID?: number }
) => Promise<any>;
};
};
};
data: {
core: any;
task: any;
jobs: Record<number, any>;
modelID: string;
modelURL: string;
embeddings: LRUCache<string, Tensor>;
Expand Down Expand Up @@ -122,9 +131,23 @@ function modelData(
}

const samPlugin: SAMPlugin = {
name: 'Segmeny Anything',
name: 'Segment Anything',
description: 'Plugin handles non-default SAM serverless function output',
cvat: {
jobs: {
get: {
async leave(
plugin: SAMPlugin,
results: any[],
query: { jobID?: number },
): Promise<any> {
if (typeof query.jobID === 'number') {
[plugin.data.jobs[query.jobID]] = results;
}
return results;
},
},
},
lambda: {
call: {
async enter(
Expand Down Expand Up @@ -159,15 +182,19 @@ const samPlugin: SAMPlugin = {
mask: number[][];
bounds: [number, number, number, number];
}> {
if (!plugin.data.task || plugin.data.task.id !== taskID) {
[plugin.data.task] = await plugin.data.core.tasks.get({ id: taskID });
}
const { height: imHeight, width: imWidth } = await plugin.data.task.frames.get(frame);
const key = `${taskID}_${frame}`;
if (model.id !== plugin.data.modelID) {
return result;
}

const job = Object.values(plugin.data.jobs)
.find((_job) => _job.taskId === taskID);
if (!job) {
throw new Error('Could not find a job corresponding to the request');
}

const { height: imHeight, width: imWidth } = await job.frames.get(frame);
const key = `${taskID}_${frame}`;

if (result) {
const bin = window.atob(result.blob);
const uint8Array = new Uint8Array(bin.length);
Expand Down Expand Up @@ -238,7 +265,7 @@ const samPlugin: SAMPlugin = {
},
data: {
core: null,
task: null,
jobs: {},
modelID: 'pth-facebookresearch-sam-vit-h',
modelURL: '/api/lambda/sam_detector.onnx',
embeddings: new LRUCache({
Expand Down

0 comments on commit 7f03eb0

Please sign in to comment.