Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added validation of duration field for change:frame #7211

Merged
merged 12 commits into from
Dec 8, 2023
Merged

Conversation

bsekachev
Copy link
Member

@bsekachev bsekachev commented Dec 1, 2023

Motivation and context

Fix exceptions with 500 code, like this in our logging system:

Error: KeyError: 'duration'

Error count: 131

Stack:
Traceback (most recent call last):
File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/django/cvat/apps/events/views.py", line 32, in create
serializer.is_valid(raise_exception=True)
File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 227, in is_valid
self._validated_data = self.run_validation(self.initial_data)
File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 426, in run_validation
value = self.to_internal_value(data)
File "/home/django/cvat/apps/events/serializers.py", line 59, in to_internal_value
event_duration += datetime.timedelta(milliseconds=event["duration"])
KeyError: 'duration'

How has this been tested?

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)
  • I have increased versions of npm packages if it is necessary
    (cvat-canvas,
    cvat-core,
    cvat-data and
    cvat-ui)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@bsekachev bsekachev changed the title Added duration validation Added validation of duration field for change:frame Dec 1, 2023
Copy link

codecov bot commented Dec 1, 2023

Codecov Report

Merging #7211 (de93453) into develop (52d1650) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #7211      +/-   ##
===========================================
- Coverage    81.35%   81.33%   -0.02%     
===========================================
  Files          366      366              
  Lines        39357    39358       +1     
  Branches      3644     3645       +1     
===========================================
- Hits         32017    32010       -7     
- Misses        7340     7348       +8     
Components Coverage Δ
cvat-ui 74.99% <100.00%> (-0.05%) ⬇️
cvat-server 87.09% <100.00%> (+<0.01%) ⬆️

def validate(self, data):
for event in data["events"]:
if event["scope"] in self._COLLAPSED_EVENT_SCOPES:
if not isinstance(event.get("duration"), int):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something seems wrong here. EventSerializer.duration has a default, so it should not be possible for it to be anything other than an int:

>>> s = EventSerializer(data={'scope': 'change:frame', 'timestamp': '2023-12-06T10:47:14.085178'})
>>> s.is_valid()
True
>>> s.validated_data['duration']
0

Are we skipping EventSerializer's validation somehow?

Copy link
Member Author

@bsekachev bsekachev Dec 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. yes, it has

default implementation for validate in Serializer class is:

    def validate(self, attrs):
        return attrs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate only handles custom validation, so it makes sense that it does nothing by default. The per-field validation is supposed to happen somewhere inside is_valid.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, perhaps we only need to keep default value in to_internal_value
Check is not necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, now it uses default value from nested class

@bsekachev bsekachev changed the title Added validation of duration field for change:frame [WIP] Added validation of duration field for change:frame Dec 7, 2023
@bsekachev bsekachev changed the title [WIP] Added validation of duration field for change:frame Added validation of duration field for change:frame Dec 7, 2023
@SpecLad SpecLad merged commit 6cfab41 into develop Dec 8, 2023
34 checks passed
@SpecLad SpecLad deleted the bs/duration_val branch December 8, 2023 11:54
@cvat-bot cvat-bot bot mentioned this pull request Dec 11, 2023
amjadsaadeh pushed a commit to amjadsaadeh/cvat that referenced this pull request Dec 14, 2023
Fix exceptions with 500 code, like this in our logging system: 
```
Error: KeyError: 'duration'

Error count: 131

Stack:
Traceback (most recent call last):
File "/opt/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/home/django/cvat/apps/events/views.py", line 32, in create
serializer.is_valid(raise_exception=True)
File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 227, in is_valid
self._validated_data = self.run_validation(self.initial_data)
File "/opt/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 426, in run_validation
value = self.to_internal_value(data)
File "/home/django/cvat/apps/events/serializers.py", line 59, in to_internal_value
event_duration += datetime.timedelta(milliseconds=event["duration"])
KeyError: 'duration'
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants