From e43a17df4a1b65a39bc2f98b0c14b0a5522e777b Mon Sep 17 00:00:00 2001 From: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> Date: Fri, 29 May 2020 18:11:47 +0300 Subject: [PATCH] Release 1.0.0 (#1335) * Added button to cancel started automatic annotation (#1198) * [WIP] Cuboid feature user guide (#1218) * Initial cuboid description * Added Gifs * Added gifs to descriptions * Formatting fixes * Codacy Fixes * Az/fix annotation dump upload (#1229) * fixed upload annotation in case of frame step != 1 * fixed upload annotation in case of attribute value is empty * Setup tag forward to the state * React UI: Added shortcuts (#1230) * [Datumaro] Label remapping transform (#1233) * Add label remapping transform * Apply transforms before project saving * Refactor voc converter * [Datumaro] Optimize mask operations (#1232) * Optimize mask to rle * Optimize mask operations * Fix dm format cmdline * Use RLE masks in datumaro format * Added tag support in new UI (without canvas drawing) * merge fix * Fixed copying/pasting actions * Deleted unused objects * [Datumaro] Dataset format auto detection (#1242) * Add dataset format detection * Add auto format detection for import * Split VOC extractor * Some debian package manager tweaks (#1235) * Some debian package manager tweaks By default, Ubuntu or Debian based "apt" or "apt-get" system installs recommended but not suggested packages . By passing "--no-install-recommends" option, the user lets apt-get know not to consider recommended packages as a dependency to install. This results in smaller downloads and installation of packages . Refer to blog at [Ubuntu Blog](https://ubuntu.com/blog/we-reduced-our-docker-images-by-60-with-no-install-recommends) . * doc: fix description of attribute CVAT XML format (#1168) * Fixed security issues in Datumaro (#1244) * Fixed security issues reported by bandit. * Fixed voc_format extractor * Sorted requirements, added a comment, removed nosec for exec. * Fix copying and creating tags * fixed git sync app (#1247) * fixed git sync app * removed shell=True for subprocess call * Fixed tags color changing and hiding * Fixed filters with tags * wip * PR fixed * Styles fixed * PR fixed * Update develop from release-0.6.0 branch (#1266) * temp * React UI: Attribute annotation mode (#1255) * Done main work * Fixed mount/unmount for canvas wrapper * Refactoring, added filters * Added missed file * Removed unnecessary useEffect * Removed extra code * Max 9 attributes, inputNumber -> Input in aam * Added blur * Renamed component * Fixed condition when validate number attribute * Some minor fixes * Fixed hotkeys config * Fixed canvas zoom * Improved behaviour of number & text * Fixed attributes switching order * Fix tags * Fixed interval * Installation issues for development environment (#1280) * Installation issues * Added ffmpeg * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui (#1270) * Bump acorn from 6.3.0 to 6.4.1 in /cvat-ui Bumps [acorn](https://github.com/acornjs/acorn) from 6.3.0 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](https://github.com/acornjs/acorn/compare/6.3.0...6.4.1) Signed-off-by: dependabot[bot] * Updated CHANGELOG.md Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev * Bump acorn from 6.2.1 to 6.4.1 in /cvat-canvas (#1281) Bumps [acorn](https://github.com/acornjs/acorn) from 6.2.1 to 6.4.1. - [Release notes](https://github.com/acornjs/acorn/releases) - [Commits](https://github.com/acornjs/acorn/compare/6.2.1...6.4.1) Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Use source label map for voc export (#1276) * Use source label map for voc export * Add line to changelog * [Datumaro] Fix frame matching in video annotations import (#1274) * Add extra frame matching way for videos * Add line to changelog * [Datumaro] Allow empty COCO dataset export (#1272) * Allow empty dataset export in coco * Add line to changelog Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * [Datumaro] Fix occluded and z_order attributes export (#1271) * Fix occluded and z_order attributes export * Add line to changelog Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Fix LabelMe format (#1260) * Fix labelme filenames * Change module path * Add tests for LabelMe * Update test * Fix test * Add line in changelog * Added point deletion context menu * React UI: Added logging (#1288) * OpenVino 2020 (#1269) * added support for OpenVINO 2020 * fixed dextr and tf_annotation Co-authored-by: Andrey Zhavoronkov * fixed point context menu for rectangles * Add recursive importers (#1290) * [Datumaro] MOT format (#1289) * Add mot format base * Add mot format * Extract common code * [Datumaro] LabelMe format (#1293) * Little refactoring * Add LabelMe format * [Datumaro] Update LabelMe format (#1296) * Little refactoring * Add LabelMe format * Add usernames * Update tests * Add extractor test * Add information about v0.6.1 release. * React UI: Better exception handling (#1297) * Fixed context menu on ubuntu * Fixed deleting of the latest point * fixes * Fix attributes with spaces in names (#1305) * fixed PR * [Datumaro] Fix image merging (#1301) * Always merge images for own dataset * Fix codacy * Validation for frame input value * Fixed UI fail when write characters in auto save interval input * Fixed input numbers in player settings * Fixed ui failing in propagate confirmation * Fixed latest input numbers, removed extra code, fixed typings * Fix navigation * Added undopoint in editing * Fixed: Could not receive frame (after merge on the latest frame) * Removed extra action dispatching * Which -> button property * Fixed: Inconsistent labels between UI and CLI/API * Fixed resize on right mouse button * Fixed create object URL after first save, fixed URL itself * Undo/redo returns frame where was a change (as it was done in previous version) * Fixed unit tests * [Datumaro] Extract common extractor functionality (#1319) * Extract common extractor functionality * Simplify coco extractor * Fix tfrecord * Fix AWS deployment (#1316) * Don't use antd less (big memory consumtion during the build process) * Fix AWS deployment guide * fix a problem with proxy and long domain names * remove sass loader for antd * Removed less and less-loader. * Simplified webpack config. * Data streaming using chunks (#1007) Huge feature (200+ commits from different developers). It completely changes layout of data (please expect very long DB migration process if you have a lot of tasks). The primary idea is to send data as zip chunks (e.g. 36 images in one chunk) or encoded video chunks and decode them on the client side. It helps to solve the problem with latency when you try to view a separate frame in the UI quickly (play mode). Another important feature of the patch is to provide access to the original images. Thus for annotations the client uses compressed chunks but if you want to export a dataset Datumaro will use original chunks (but video will be decoded with original quality and encoded with maximum/optimal quality in any case). * Shortcuts keymaps moved to state * Titles for objects in side menu * Fixed bug in menu * Titles in attribute annotations mode * Controls panel titles * Titles for object list header * Minor fixes * Added tooltips in top bar * Added settings tooltip * Optimized patch * Typos * Fix a problem with known hosts inside git app (cannot clone a repo from github.com) (#1330) * Fixed zOrder range computing in case when there are tags * Small preview and progress (#1331) * Reduce preview size (untested) * Fix tests * Improve media readers (untested) * fixed migration * fixed frame provider * fixed preview save * fixed stop frame * handle duration == None * codacy * added missed import * unified iteration over frames for media readers and fixed corner case when user specify stop_frame = 0 Co-authored-by: Nikita Manovich * Move annotation formats to dataset manager (#1256) * Move formats to dataset manager * Unify datataset export and anno export implementations * Add track_id to TrackedShape, export tracked shapes * Replace MOT format * Replace LabelMe format * Add new formats to dm * Add dm tests * Extend TrackedShape * Enable dm test in CI * Fix tests * Add import * Fix tests * Fix mot track ids * Fix mot format * Update attribute logic in labelme tests * Use common code in yolo * Put datumaro in path in settings * Expect labels file in MOT next to annotations file * Add MOT format description * Add import * Add labelme format description * Linter fix * Linter fix2 * Compare attributes ordered * Update docs * Update tests * Az/fix migration (#1333) * fixed migration because readers interface changed * fixed tests * Fixed escape in draw * Insert multiple shapes * Added dialog window with some help info about filters * Typos in doc * Add missing information into changelog. * Add information for next release + update version. * Updated changelog * Updated changelog * Special behaviour for the attribute value __undefined__ * Fixed license year * No break space const * Updated changelog * Fixed year in license headers * Implementation of bitmap in client * Updated changelog * Z-layer support * Fixed settings after reopen a job * Do not show invisible objects on bitmap * Fix point interpolation (#1344) * Extend formats tests with different track types * Add unordered list comparison * Skip empty list comparison * fix * fix * Reproduce problem * Fix point interpolation for single point * undo rest api refactor * Added button to reset color settings * Updated changelog * Added option to display shape text always * Updated changelog * Hidden/outside fix * Fixed screen scaling * fixed dump error after moving format files (#1342) * fixed dump error after moving format files * updated changelog * Az/fix dextr (#1348) * Fixed dextr_segmentation app * updated changelog Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Disabled option by default * Fixed typos in interface * Increase preview size till 256, 256. Previous preview size was not optimal and led to a blurred image due to too small size. * Add a line into CHANGELOG.md about the change. * Refactor frame provider (#1355) * Refactor frame provider * fix * Add CODEOWNERS file (#1360) * Add CODEOWNERS file * Removed the outdated file. See https://github.com/opencv/cvat/graphs/contributors * Change codeowners * Add pull request and issue templates (#1359) * Add initial version of pull request template * Fix links * Fix codacy issues * Slightly improve titles of sections * Add a note about strikethough for the checklist. * Fix progress of a pull request (each checkbox is an issue) * Add the license header, checkboxes about the license. * Updated the license * Update the license to met https://github.com/licensee/licensee/blob/master/vendor/choosealicense.com/_licenses/mit.txt restrictions. * Fix the pull request template name * Make explaination text as comments (it will be visible when you edit the PR message) * Add initial version of the issue template. * Batch of fixes (#1370) * Some margins were change to paddings * Removed extra selected * Fix: added outside shapes when merge polyshapes * Fixed double scroll bars * Updated canvas table * Fixed setup methodf * Disabled change frame during drag, resize and editing * Fixed: hidden points are visible * Fixed: Merge is allowed for points, but clicks on points conflict with frame dragging logic * Fixed: do not filter removed objects * Updated CHANGELOG.md * Couple of headers updated * Added missed fields in exception logs (#1372) * Simplified codeowners file (#1376) * Fixed points visibility when go between frames * React UI: Added message when share is empty or not mounted (#1373) * Added message when share is empty or not mounted * Updated changelog * Update CHANGELOG.md Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Save full image paths in coco (#1381) * Add chunk iterator cache to frame provider (#1367) * Add chunk iterator cache * fix * Update item "Creating an annotation task" in User Guide (#1363) * Az/fix remote files (#1392) * fixed task creation from remote files * Update CHANGELOG.md * [Datumaro] Fix COCO keypoint export bug (#1388) * Instructions on using HTTPS (#1357) * Fix label comparison in voc format (#1382) * Update item "Interface of the annotation tool" in User Guide (#1386) * React UI: Batch of fixes (#1383) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Update item "Basic navigation" in User Guide and bug fix in user_guide.md (#1395) * Fix git app paths (#1400) * changed paths for the git repos * Update CHANGELOG.md * updated licence header * React UI: Displaying public ssh keys in UI (#1375) * Updated changelog * Typos * Batch of fixes (#1403) * Fixed bug when job cannot be opened * Fixed bug when deactivated shape is still highlighted * Fixed Error: 'AttributeError: 'tuple' object has no attribute 'read' * Fixed: wrong semi-automatic segmentation near edges of an image * Updated changelog Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * React UI: Semi-automatic segmentation (#1398) * implemented checked * Implemented plugin * Added dialog windows * Updated changelo * Added cancel request * React UI: Automatic bordering for polygons and polylines during drawing/editing (#1394) * Fixed: cannot read property 'set' of undefined * Fixed UI failing: save during drag/resize * Fixed multiple saving (shortcut sticking) * Undo/redo fixed * Allowed one interpolated point * Fixed API reaction when repository synchronization is failed * Updated changelog * Auto bordering feature * Some fixes, added shortcuts * Fixed draw when start with one of supporting point * React UI: batch of fixes (#1404) * React UI has become a primary UI * Temporary disabled cuboid in cvat-core * Fixed: Failed to execute removChild.. (#1405) * Update CHANGELOG.md * Updated CHANGELOG (new version will be 1.0.0-beta.1) * Update CHANGELOG.md (next version is 1.0.0-beta.2) * Updated version of CVAT server till beta.2 * Fixed auto annotation, tf annotation and auto segmentation apps (#1409) * fixed code that uses FrameProvider, as the interface has changed * Update CHANGELOG.md * Update item "Types of shapes" (#1401) * React UI: ReID algorithm (#1406) * Initial commit * Connected storage * Added core API method * Done implementation * Removed rule * Removed double cancel * Updated changelog * Fixed: Cannot read property toFixed of undefined * Update CHANGELOG.md * Improve PR template (#1427) * Simplified PR template. * Remove a new line to make codacy happy. * fix: OSError:broken data stream (#1430) * [Datumaro] Fix duplicating keypoints in COCO export (#1435) * React UI: Fixed typos in remove annotations confirmation (#1450) * React UI: Batch of fixes (#1445) * Hide functionality (H) doesn't work * The highlighted attribute doesn't correspond to the chosen attribute in AAM * Inconvinient image shaking while drawing a polygon (hold Alt key during drawing/editing/grouping to drag an image) * Filter property "shape" doesn't work and extra operator in description * Block of text information doesn't disappear after deactivating for locked shapes * Annotation uploading fails in annotation view * UI freezes after canceling pasting with escape Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Update item "Annotation mode (basics)" in User Guide (#1412) * Update item "annotation mode (basics)" in User Guide * Replacing a gif017 with an image * React UI: Added client versioning (#1448) * Adjusted antd import * Wrapped core and canvas * Added versioning * Updated changelog, adjusted installation guide a bit * Update item "Interpolation mode (basics)" in User Guide (#1455) * Update item interpolation mode in user guide fix typos and contents * Fix a typo * Fix a typo in the filename Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * React UI: batch of fixes (#1462) * CVAT new UI: add arrows on a mouse cursor * Delete point bug (in new UI) * fix auto annotation to not eat all RAM (#1328) Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Fixed versioning for node 8 (#1466) * Update items from AAM (basics) to Vocabulary in User Guide (#1459) * Update items from AAM (basics) to Vocabulary * Delete unused images and gif * Fix apache startup (#1467) * Fix Network Error after PC Restart (#1035) * Update CHANGELOG.md * Remove deprecated utils (#1477) * removed deprecated convert scripts * updated changelog * Fixed 'Open task' button doesn't work (#1474) * Fixed uploading track annotations for multi-segment tasks (#1396) * fixed uploading annotation for overlapped segments * fixed dump of tracks for multisegment task * Update CHANGELOG.md * fixed comments * fixed comments * Update cvat/apps/engine/data_manager.py Co-Authored-By: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * drop start shapes with outside==True for splitted track * code cleanup * fixed typo * fix Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * React UI: cuboids (#1451) * Update changelog. * Update CHANGELOG and the version of CVAT server. * session.annotations.put() returns indexes of added objects (#1493) * session.annotations.put() returns indexes of added objects * Updated changelog * Updated README.md files (added info about versioning) (#1490) * Updated README.md files (added info about versioning) * Typos * Add coverage for python (#1483) * Fix coverage merging (#1504) * Updating instructions to serve Swagger documentation (#1502) * Update items Workspace and Types of shapes in User Guide (#1497) * Merge annotations and dataset_manager apps (#1352) * Fix coverage measurement (#1516) * fixed linter issues and store credentials cookie in the session object (#1526) * fixed issues * fixed issues and stored credentials cookies inside the session * fixed tests * React UI: batch of fixes (#1525) * Update item Settings in User Guide (#1508) * React UI: cookie policy drawer (#1511) * fixed linter issues (#1538) * fixed false tag activation (#1541) * Added item with npm package version increasing to the PR template (#1542) * [Datumaro] Fix coco import conflict with labels (#1548) * [Datumaro] Change alignment in mask parsing (#1547) * Include empty images in exported annotations (#1479) * Update item Bottom panel in User Guide (#1509) * Update item Bottom panel in User Guide rename the "Bottom panel" to the "Top panel" * Fixed typos, remove trailing spaces in user_guide.md * Fix user_guide.md and update image051.jpg * Ability to configure user agreements for the register user form (#1464) * Update item Side panel in User guide (#1513) * Update item Side panel in User guide * Removed trailing spaces in an Object sidebar item * Update cvat/apps/documentation/user_guide.md * fix typo * missing image correction Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Layout styles fixes * React UI: cuboid interpolation and cuboid drawing from rectangles (#1560) * Added backend cuboid interpolation and cuboid drawing from rectangles * Added CHANELOG.md * Fixed cuboid front edges stroke width * PR fixes * Fixed auto_segmentation app (#1562) * disabled tf eager execution for auto_segmentation * Update CHANGELOG.md * Add VOC grayscale masks test and documentation (#1576) * Add a test for unpainted masks * Update format documentation * [Datumaro] Fix mask to polygons warning (#1581) * Fix message, add test * update changelog * Fix cuboid conversion (#1577) * Fix cuboid conversion * update changelog Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * [Datumaro] Simplify log level setting (#1583) * Simplify loglevel setting * update changelog * Fixed git synchronization (#1582) * fixed git synchronization * Update CHANGELOG.md Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Fixed dextr, fixed moving of the canvas (#1573) * Fixed dextr, fixed moving of the canvas * Updated CONTRIBUTUNG.md, updated version Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * Added the ability to configure custom pageViewHit (may useful for web analytics) (#1566) * added the ability to configure custom pageViewHit (may useful for web analytics) * updated version and changelog * fixed comments * cvat-ui minor v++ * subscribe on history updates in the root component * updated Online Demo section of Readme (#1588) * updated Online Demo section of Readme * Change content of "online demo" section Co-authored-by: Nikita Manovich * Fixed task creation for videos with uneven dimensions. (#1594) * used yuv420p format for compressed and original chunks * updated changelog * added settings to reduce access to analytics component (#1592) * added settings to reduce access to analytics component * updated CHANGELOG * fixed typo * Add item Controls sidebar in User Guide (#1510) * Update text, images and gif in user_guide.md (#1558) * Update user_guide.md, images and gif (#1556) Co-authored-by: Nikita Manovich <40690625+nmanovic@users.noreply.github.com> * update user_guide.md, images and gifs * Delete gif013 * add gif013 with correct name * Use z_order as a class property (#1589) * Use z_order as a class property * Fix z_order use in voc * Update changelog * Update item Annotation with polylines (#1596) * update user_guide.md and images * fix uppercase letters in images path in user_guide.md and remove trailing spaces * delete images and gifs containing uppercase letters in the name * add images with correct names * fix image paths in user_guide.md * Delete image133 * add image133 with correct name * Fix example yaml format (#1603) * delete duplicate item in user_guide.md (#1607) * update user_guide.md and add image (#1604) * update gifs, images and user_guide.md (#1605) * fix analytics permissions (#1608) * Update item Annotation with cuboids (#1598) * Update item Annotation with polygons in User guide v2 (#1612) * update user_guide.md * update images and gifs * Update Filter, Analytics and Shortcuts items in User Guide (#1606) * update user_guide.md and images * fix user_guide.md * delete unused image * delete unused images * Fix duplicate item in User Guide (#1617) * remove item Annotation with Auto Segmentation * fix link in user_guide.md * delete unused images * Slightly improve changelog * Update CVAT version Co-authored-by: Boris Sekachev <40690378+bsekachev@users.noreply.github.com> Co-authored-by: Tritin Truong Co-authored-by: Andrey Zhavoronkov <41117609+azhavoro@users.noreply.github.com> Co-authored-by: Dmitry Kalinin Co-authored-by: zhiltsov-max Co-authored-by: Pratik Raj Co-authored-by: Mathis Chenuet Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Boris Sekachev Co-authored-by: Ben Hoff Co-authored-by: Andrey Zhavoronkov Co-authored-by: TOsmanov <54434686+TOsmanov@users.noreply.github.com> Co-authored-by: ranko r sredojevic Co-authored-by: Thomas Albrecht Co-authored-by: Johannes222 Co-authored-by: Gururaj Jeerge Co-authored-by: timurx.osmanov Co-authored-by: YutaYamazaki <37947061+yutayamazaki@users.noreply.github.com> --- CHANGELOG.md | 59 ++--- cvat-core/src/annotations-collection.js | 3 + cvat-core/src/server-proxy.js | 7 + cvat/__init__.py | 2 +- cvat/apps/documentation/installation.md | 301 ++++++++++++++---------- cvat/apps/documentation/user_guide.md | 13 +- cvat_proxy/conf.d/cvat.conf.template | 4 + 7 files changed, 222 insertions(+), 167 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 01d1014746c9..600c9676b2e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,43 +4,39 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [1.0.0] - Unreleased +## [1.0.0] - 2020-05-29 ### Added -- cvat-ui: added cookie policy drawer for login page () -- Added `datumaro_project` export format (https://github.com/opencv/cvat/pull/1352) -- Ability to configure user agreements for the user registration form (https://github.com/opencv/cvat/pull/1464) -- Added cuboid interpolation and cuboid drawing from rectangles () -- Ability to configure custom pageViewHit, which can be useful for web analytics integration (https://github.com/opencv/cvat/pull/1566) -- Ability to configure access to the analytics page based on roles (https://github.com/opencv/cvat/pull/1592) +- cvat-ui: cookie policy drawer for login page () +- `datumaro_project` export format () +- Ability to configure user agreements for the user registration form () +- Cuboid interpolation and cuboid drawing from rectangles () +- Ability to configure custom pageViewHit, which can be useful for web analytics integration () +- Ability to configure access to the analytics page based on roles () ### Changed -- Downloaded file name in annotations export became more informative (https://github.com/opencv/cvat/pull/1352) -- Added auto trimming for trailing whitespaces style enforcement (https://github.com/opencv/cvat/pull/1352) -- REST API: updated `GET /task//annotations`: parameters are `format`, `filename` (now optional), `action` (optional) (https://github.com/opencv/cvat/pull/1352) -- REST API: removed `dataset/formats`, changed format of `annotation/formats` (https://github.com/opencv/cvat/pull/1352) -- Exported annotations are stored for N hours instead of indefinitely (https://github.com/opencv/cvat/pull/1352) -- Formats: CVAT format now accepts ZIP and XML (https://github.com/opencv/cvat/pull/1352) -- Formats: COCO format now accepts ZIP and JSON (https://github.com/opencv/cvat/pull/1352) -- Formats: most of formats renamed, no extension in title (https://github.com/opencv/cvat/pull/1352) -- Formats: definitions are changed, are not stored in DB anymore (https://github.com/opencv/cvat/pull/1352) -- cvat-core: session.annotations.put() now returns identificators of added objects (https://github.com/opencv/cvat/pull/1493) -- Images without annotations now also included in dataset/annotations export (https://github.com/opencv/cvat/issues/525) -- Update https install manual. Makes it easier and more robust. Includes automatic renewing of lets encrypt certificates. - -### Deprecated -- +- Downloaded file name in annotations export became more informative () +- Added auto trimming for trailing whitespaces style enforcement () +- REST API: updated `GET /task//annotations`: parameters are `format`, `filename` (now optional), `action` (optional) () +- REST API: removed `dataset/formats`, changed format of `annotation/formats` () +- Exported annotations are stored for N hours instead of indefinitely () +- Formats: CVAT format now accepts ZIP and XML () +- Formats: COCO format now accepts ZIP and JSON () +- Formats: most of formats renamed, no extension in title () +- Formats: definitions are changed, are not stored in DB anymore () +- cvat-core: session.annotations.put() now returns ids of added objects () +- Images without annotations now also included in dataset/annotations export () ### Removed -- `annotation` application is replaced with `dataset_manager` (https://github.com/opencv/cvat/pull/1352) -- `_DATUMARO_INIT_LOGLEVEL` env. variable is removed in favor of regular `--loglevel` cli parameter (https://github.com/opencv/cvat/pull/1583) +- `annotation` application is replaced with `dataset_manager` () +- `_DATUMARO_INIT_LOGLEVEL` env. variable is removed in favor of regular `--loglevel` cli parameter () ### Fixed -- Categories for empty projects with no sources are taken from own dataset (https://github.com/opencv/cvat/pull/1352) -- Added directory removal on error during `extract` command (https://github.com/opencv/cvat/pull/1352) -- Added debug error message on incorrect XPath (https://github.com/opencv/cvat/pull/1352) -- Exporting frame stepped task (https://github.com/opencv/cvat/issues/1294, https://github.com/opencv/cvat/issues/1334) -- Fixed broken command line interface for `cvat` export format in Datumaro (https://github.com/opencv/cvat/issues/1494) -- Updated Rest API document, Swagger document serving instruction issue (https://github.com/opencv/cvat/issues/1495) +- Categories for empty projects with no sources are taken from own dataset () +- Added directory removal on error during `extract` command () +- Added debug error message on incorrect XPath () +- Exporting frame stepped task () +- Fixed broken command line interface for `cvat` export format in Datumaro () +- Updated Rest API document, Swagger document serving instruction issue () - Fixed cuboid occluded view () - Non-informative lock icon () - Sidebar in AAM has no hide/show button () @@ -60,9 +56,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Fixed an issue with `z_order` having no effect on segmentations () - Permission group whitelist check for analytics view () -### Security -- - ## [1.0.0-beta.2] - 2020-04-30 ### Added - Re-Identification algorithm to merging bounding boxes automatically to the new UI () diff --git a/cvat-core/src/annotations-collection.js b/cvat-core/src/annotations-collection.js index 8616b88781b2..654e89c5b248 100644 --- a/cvat-core/src/annotations-collection.js +++ b/cvat-core/src/annotations-collection.js @@ -96,6 +96,9 @@ case 'cuboid': trackModel = new CuboidTrack(trackData, clientID, color, injection); break; + case 'cuboid': + trackModel = new CuboidTrack(trackData, clientID, color, injection); + break; default: throw new DataError( `An unexpected type of track "${type}"`, diff --git a/cvat-core/src/server-proxy.js b/cvat-core/src/server-proxy.js index 93fba2569e83..f6db560a9a3a 100644 --- a/cvat-core/src/server-proxy.js +++ b/cvat-core/src/server-proxy.js @@ -752,6 +752,13 @@ }), writable: false, }, + + logs: { + value: Object.freeze({ + save: saveLogs, + }), + writable: false, + }, })); } } diff --git a/cvat/__init__.py b/cvat/__init__.py index 6409842f25f8..61cafddfe3db 100644 --- a/cvat/__init__.py +++ b/cvat/__init__.py @@ -4,6 +4,6 @@ from cvat.utils.version import get_version -VERSION = (1, 0, 0, 'rc', 0) +VERSION = (1, 0, 0, 'final', 0) __version__ = get_version(VERSION) diff --git a/cvat/apps/documentation/installation.md b/cvat/apps/documentation/installation.md index 0aa5227242b4..158a987b8926 100644 --- a/cvat/apps/documentation/installation.md +++ b/cvat/apps/documentation/installation.md @@ -311,55 +311,47 @@ contains a text (url for example) which is shown in the client-share browser. ### Serving over HTTPS We will add [letsencrypt.org](https://letsencrypt.org/) issued certificate to secure -our server connection. +our server connection. #### Prerequisites -We assume that +We assume that -- you have sudo access on your server machine, +- you have sudo access on your server machine, - you have an IP address to use for remote access, and - that the local CVAT installation works on your server. - + If this is not the case, please complete the steps in the installation manual first. #### Roadmap We will go through the following sequence of steps to get CVAT over HTTPS: -- Setup containers on default 80/tcp port. Checkin and then down the containers. -- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/) - webroot. +- Move Docker Compose CVAT access port to 80/tcp. +- Configure Nginx to pass one of the [ACME challenges](https://letsencrypt.org/docs/challenge-types/). - Create the certificate files using [acme.sh](https://github.com/acmesh-official/acme.sh). - Reconfigure Nginx to serve over HTTPS and map CVAT to Docker Compose port 443. #### Step-by-step instructions -##### 1. Make the proxy listen on standard port 80 and prepare nginx for the ACME challenge via webroot method - -> The configuration assumes that on the docker host there will be only one instance of the CVAT site listens for incoming connections on 80 and 443 port. Also redirecting everything that does not concern renewal of certificates to the site via secure HTTPS protocol. - -Let's assume the server will be at `my-cvat-server.org`. - -Point you shell in cvat repository directory, usually `cd $HOME/cvat`: - -Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. This file lives in the same directory as `docker-compose.yml`. - -Create enough directories for letsencrypt webroot operation and acme folder passthrougth. +##### 1. Move the CVAT access port - and restart containers with a new configuration updated in `docker-compose.override.yml` +Let's assume the server will be at `my-cvat-server.org`. ```bash -# on the docker host +# on the server +docker-compose down -# this will create ~/.acme.sh directory -curl https://get.acme.sh | sh +# add docker-compose.override.yml as per instructions below -# create a subdirs for acme-challenge webroot manually -mkdir -p $HOME/cvat/letsencrypt-webroot/.well-known/acme-challenge +docker-compose up -d ``` +Add the following into your `docker-compose.override.yml`, replacing `my-cvat-server.org` with your own IP address. +This file lives in the same directory as `docker-compose.yml`. + ```yaml -# docker-compose.override.yml +# docker-compose.override.yml version: "2.3" services: @@ -367,106 +359,177 @@ services: environment: CVAT_HOST: my-cvat-server.org ports: - - "80:80" - - "443:443" - volumes: - - ./letsencrypt-webroot:/var/tmp/letsencrypt-webroot - - /root/.acme.sh:/root/.acme.sh - + - "80:80" + cvat: environment: ALLOWED_HOSTS: '*' ``` -This will enable serving `http://my-cvat-server.org/.well-known/acme-challenge/` -route from `/var/tmp/letsencrypt-webroot` directory on the container's filesystem which is bind mounted from docker host `$HOME/cvat/letsencrypt-webroot`. That volume needed for issue and renewing certificates only. - -Update a CVAT site proxy template `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` on docker(system) host. Site config updates from this template each time `cvat_proxy` container start. +You should now see an unsecured version of CVAT at `http://my-cvat-server.org`. -Add a location to server with `server_name ${CVAT_HOST};` ahead others: +##### 2. Configure Nginx for the ACME challenge -``` - location ^~ /.well-known/acme-challenge/ { - default_type "text/plain"; - root /var/tmp/letsencrypt-webroot; - } -``` +Temporarily, enable serving `http://my-cvat-server.org/.well-known/acme-challenge/` +route from `/letsencrypt` directory on the server's filesystem. You can use the [Nginx quickstart guide](http://nginx.org/en/docs/beginners_guide.html) for reference. - ```bash -# on the docker host -docker-compose down -docker-compose up -d -``` +# cvat_proxy/conf.d/cvat.conf.template -Your server should still be visible (and unsecured) at `http://my-cvat-server.org` -but you won't see any behavior changes. +server { + listen 80; + server_name _ default; + return 404; +} -At this point your deployment is up and running, ready for run acme-challenge. +server { + listen 80; + server_name ${CVAT_HOST}; -##### 2. Setting up HTTPS with `acme.sh` helper + # add this temporarily, to pass an acme challenge + location ^~ /.well-known/acme-challenge/ { + allow all; + root /letsencrypt; + } -There are multiple approaches. First one is to use helper on docker host. + location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* { + proxy_pass http://cvat:8080; + proxy_pass_header X-CSRFToken; + proxy_set_header Host $http_host; + proxy_pass_header Set-Cookie; + } -In a our approach -* it is easier to setup automatic certificate updates and (than it can be done in the container). -* leave certificates in safe place on docker host (protect from `docker-compose down` cleanup) -* no unnecessary certificate files copying between container and host. + location / { + # workaround for match location by arguments + error_page 418 = @annotation_ui; -###### Create certificate files using an ACME challenge on docker host + if ( $query_string ~ "^id=\d+.*" ) { return 418; } -**Prepare certificates.** + proxy_pass http://cvat_ui; + proxy_pass_header X-CSRFToken; + proxy_set_header Host $http_host; + proxy_pass_header Set-Cookie; + } -Point you shell in cvat repository directory, usually `cd $HOME/cvat` on docker host. + # old annotation ui, will be removed in the future. + location @annotation_ui { + proxy_pass http://cvat:8080; + proxy_pass_header X-CSRFToken; + proxy_set_header Host $http_host; + proxy_pass_header Set-Cookie; + } +} +``` -> Certificate issue and updates should be on docker host in this approach. +Now create the `/letsencrypt` directory and mount it into `cvat_proxy` container. +Edit your `docker-compose.override.yml` to look like the following: + +```yaml +# docker-compose.override.yml +version: "2.3" -Let’s Encrypt provides rate limits to ensure fair usage by as many people as possible. They recommend utilize their staging environment instead of the production API during testing. So first try to get a test certificate. +services: + cvat_proxy: + environment: + CVAT_HOST: my-cvat-server.org + ports: + - "80:80" + volumes: + - ./letsencrypt:/letsencrypt -``` -~/.acme.sh/acme.sh --issue --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug + cvat: + environment: + ALLOWED_HOSTS: '*' ``` -> Debug note: nginx server logs for cvat_proxy are not saved in container. You shall see it at docker host by with: `docker logs cvat_proxy`. +Finally, create the directory and restart CVAT. -If certificates is issued a successful we should test a renew: +```bash +# in the same directory where docker-compose.override.yml lives +mkdir -p letsencrypt/.well-known/acme-challenge +docker-compose down +docker-compose up -d ``` -~/.acme.sh/acme.sh --renew --force --staging -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug -``` -If success: -* remove test certificate -* issue a production certificate -* create a cron job for user (`crontab -e`). +Your server should still be visible (and unsecured) at `http://my-cvat-server.org` +but you won't see any behavior changes. -``` -~/.acme.sh/acme.sh --remove -d my-cvat-server.org --debug -rm -r /root/.acme.sh/my-cvat-server.org +##### 3. Create certificate files using an ACME challenge -~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w $HOME/cvat/letsencrypt-webroot --debug +At this point your deployment is running. -~/.acme.sh/acme.sh --install-cronjob +```bash +admin@tempVM:~/cvat$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +0a35cd127968 nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp, 0.0.0.0:8080->80/tcp cvat_proxy +b85497c44836 cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui +d25a00475849 cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat +6353a43f55c3 redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis +52009636caa8 postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db ``` -Down the cvat_proxy container for setup https with issued certificates. +We will attach `cvat_proxy` container to run `acme.sh` scripts. ```bash -docker stop cvat_proxy +admin@tempVM:~/cvat$ docker exec -ti cvat_proxy /bin/sh + +# install some missing software inside cvat_proxy +/ # apk add openssl curl +/ # curl https://get.acme.sh | sh +/ # ~/.acme.sh/acme.sh -h +[... many lines ...] + +/ # ~/.acme.sh/acme.sh --issue -d my-cvat-server.org -w /letsencrypt +[Fri Apr 3 20:49:05 UTC 2020] Create account key ok. +[Fri Apr 3 20:49:05 UTC 2020] Registering account +[Fri Apr 3 20:49:06 UTC 2020] Registered +[Fri Apr 3 20:49:06 UTC 2020] ACCOUNT_THUMBPRINT='tril8-LdJgM8xg6mnN1pMa7vIMdFizVCE0NImNmyZY4' +[Fri Apr 3 20:49:06 UTC 2020] Creating domain key +[ ... many more lines ...] +[Fri Apr 3 20:49:10 UTC 2020] Your cert is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer +[Fri Apr 3 20:49:10 UTC 2020] Your cert key is in /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key +[Fri Apr 3 20:49:10 UTC 2020] The intermediate CA cert is in /root/.acme.sh/my-cvat-server.org/ca.cer +[Fri Apr 3 20:49:10 UTC 2020] And the full chain certs is there: /root/.acme.sh/my-cvat-server.org/fullchain.cer + +/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer /letsencrypt/certificate.cer +/ # cp ~/.acme.sh/my-cvat-server.org/my-cvat-server.org.key /letsencrypt/certificate.key +/ # cp ~/.acme.sh/my-cvat-server.org/ca.cer /letsencrypt/ca.cer +/ # cp ~/.acme.sh/my-cvat-server.org/fullchain.cer /letsencrypt/fullchain.cer +/ # exit +admin@tempVM:~/cvat$ ls letsencrypt/ +ca.cer certificate.cer certificate.key fullchain.cer +admin@tempVM:~/cvat$ mkdir cert +admin@tempVM:~/cvat$ mv letsencrypt/* ./cert ``` -**Reconfigure nginx for use certificates.** +##### 4. Reconfigure Nginx for HTTPS access -Bring the configuration file `$HOME/cvat/cvat_proxy/conf.d/cvat.conf.template` to the following form: +Update Docker Compose configuration to mount the certificate directory. -* add location with redirect `return 301` to 80/tcp server. -* change listen to `listen 443 ssl;` in main configururation server -* add ssl certificates options and secure them. +```yml +# docker-compose.override.yml +version: "2.3" -Example of configuration file: +services: + cvat_proxy: + environment: + CVAT_HOST: my-cvat-server.org + ports: + - "443:443" + volumes: + - ./letsencrypt:/letsencrypt + - ./cert:/cert:ro # this is new + cvat: + environment: + ALLOWED_HOSTS: '*' ``` + +Also, reconfigure Nginx to use `443/tcp` and point it to the new keys. + +```bash server { listen 80; server_name _ default; @@ -474,47 +537,16 @@ server { } server { - listen 80; - server_name ${CVAT_HOST}; - - location ^~ /.well-known/acme-challenge/ { - default_type "text/plain"; - root /var/tmp/letsencrypt; - } - - location / { - return 301 https://$server_name$request_uri; - } -} - -server { - listen 443 ssl; - ssl_certificate /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.cer; - ssl_certificate_key /root/.acme.sh/my-cvat-server.org/my-cvat-server.org.key; - ssl_trusted_certificate /root/.acme.sh/my-cvat-server.org/fullchain.cer; - - # security options - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_prefer_server_ciphers on; - ssl_stapling on; - ssl_session_timeout 24h; - ssl_session_cache shared:SSL:2m; - ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA:!3DES'; - - # security headers - add_header X-Frame-Options "SAMEORIGIN" always; - add_header X-XSS-Protection "1; mode=block" always; - add_header X-Content-Type-Options "nosniff" always; - add_header Referrer-Policy "no-referrer-when-downgrade" always; - add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always; - add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always; - + listen 443 ssl; server_name ${CVAT_HOST}; proxy_pass_header X-CSRFToken; proxy_set_header Host $http_host; proxy_pass_header Set-Cookie; + ssl_certificate /cert/certificate.cer; + ssl_certificate_key /cert/certificate.key; + location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* { proxy_pass http://cvat:8080; } @@ -538,8 +570,35 @@ server { } ``` -Start cvat_proxy container with https enabled. +Finally, restart your service. +```bash +admin@tempVM:~/cvat$ docker-compose down +Stopping cvat_proxy ... done +Stopping cvat_ui ... done +Stopping cvat ... done +Stopping cvat_db ... done +Stopping cvat_redis ... done +Removing cvat_proxy ... done +Removing cvat_ui ... done +Removing cvat ... done +Removing cvat_db ... done +Removing cvat_redis ... done +Removing network cvat_default +admin@tempVM:~/cvat$ docker-compose up -d +Creating network "cvat_default" with the default driver +Creating cvat_db ... done +Creating cvat_redis ... done +Creating cvat ... done +Creating cvat_ui ... done +Creating cvat_proxy ... done +admin@tempVM:~/cvat$ docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +71464aeac87c nginx:stable-alpine "/bin/sh -c 'envsubs…" About a minute ago Up About a minute 0.0.0.0:443->443/tcp, 0.0.0.0:8080->80/tcp cvat_proxy +8428cfbb766e cvat_cvat_ui "nginx -g 'daemon of…" About a minute ago Up About a minute 80/tcp cvat_ui +b5a2f78689da cvat "/usr/bin/supervisord" About a minute ago Up About a minute 8080/tcp, 8443/tcp cvat +ef4a1f47440f redis:4.0-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 6379/tcp cvat_redis +7803bf828d9f postgres:10-alpine "docker-entrypoint.s…" About a minute ago Up About a minute 5432/tcp cvat_db ``` -docker start cvat_proxy -``` \ No newline at end of file + +Now you can go to `https://my-cvat-server.org/` and verify that you are using an encrypted connection. diff --git a/cvat/apps/documentation/user_guide.md b/cvat/apps/documentation/user_guide.md index 3863aeefc74f..0a01c1d0df86 100644 --- a/cvat/apps/documentation/user_guide.md +++ b/cvat/apps/documentation/user_guide.md @@ -17,18 +17,10 @@ - [Workspace](#workspace) - [Settings](#settings) - [Top Panel](#top-panel) - - [Menu button](#menu-button) - - [Save Work](#save-work) - - [Undo-redo buttons](#undo-redo-buttons) - - [Player](#player) - - [Fullscreen Player](#fullscreen-player) - - [Info](#info) - - [UI switcher](#ui-switcher) - [Controls sidebar](#controls-sidebar) - [Objects sidebar](#objects-sidebar) - [Objects](#objects) - [Labels](#labels) - - [Appearance](#appearance) - [Shape mode (advanced)](#shape-mode-advanced) - [Track mode (advanced)](#track-mode-advanced) - [Attribute annotation mode (advanced)](#attribute-annotation-mode-advanced) @@ -39,10 +31,7 @@ - [Points in shape mode](#points-in-shape-mode) - [Linear interpolation with one point](#linear-interpolation-with-one-point) - [Annotation with cuboids](#annotation-with-cuboids) - - [Creating the cuboid](#creating-the-cuboid) - - [Editing the cuboid](#editing-the-cuboid) - - [Annotation with Auto Segmentation](#annotation-with-auto-segmentation) - - [Annotation with Tags](#annotation-with-tags) + - [Annotation with tags](#annotation-with-tags) - [Automatic annotation](#automatic-annotation) - [Shape grouping](#shape-grouping) - [Filter](#filter) diff --git a/cvat_proxy/conf.d/cvat.conf.template b/cvat_proxy/conf.d/cvat.conf.template index b5d9c760cbe2..fd2d336787a6 100644 --- a/cvat_proxy/conf.d/cvat.conf.template +++ b/cvat_proxy/conf.d/cvat.conf.template @@ -11,6 +11,10 @@ server { proxy_set_header Host $http_host; proxy_pass_header Set-Cookie; + proxy_pass_header X-CSRFToken; + proxy_set_header Host $http_host; + proxy_pass_header Set-Cookie; + location ~* /api/.*|git/.*|tensorflow/.*|auto_annotation/.*|analytics/.*|static/.*|admin|admin/.*|documentation/.*|dextr/.*|reid/.* { proxy_pass http://cvat:8080; }