Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements 'Jump frame by frame' in video #823

Merged
merged 24 commits into from
Jun 18, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
61f9fa9
Add "show previous frame" and "show next frame" buttons in video navi…
ToukL Apr 11, 2024
b978231
Add a supportsJumpByFrame attribute which checks for browser compatib…
ToukL May 16, 2024
2ecd78e
Merge branch 'biigle:master' into jump-frame-by-frame
ToukL May 16, 2024
fc236c2
Merge branch 'jump-frame-by-frame' of https://github.com/ToukL/biigle…
ToukL May 16, 2024
beeeae4
Fix coding style issue
ToukL May 16, 2024
2f2f47e
Merge branch 'biigle:master' into jump-frame-by-frame
ToukL May 21, 2024
b8a2974
Adapt shortcuts and icons depending on enableJumpByFrame feature
ToukL May 21, 2024
9efd6e4
Update manual for jump by frame feature
ToukL May 21, 2024
cee1f46
Prevent double calls to prev/next frame by calling seek method
ToukL May 22, 2024
8d108ca
Merge branch 'master' into jump-frame-by-frame
ToukL May 23, 2024
cc83c27
Fix coding issues
ToukL May 23, 2024
ddbdb83
Move prev/next video buttons to separate button group
ToukL May 23, 2024
2937e75
Move keyboard shortcuts on/off to a separate method
ToukL May 23, 2024
a31095e
Remove unnecessary spaces
ToukL May 23, 2024
5324956
Add arrows icons to jump by frame section in manual
ToukL May 23, 2024
f9c6020
move jump frame button to the right
ToukL Jun 6, 2024
eff1d3e
Add small changes in manual
ToukL Jun 6, 2024
fc49c77
Fix bug on video edge conditions and infinite seeking state
ToukL Jun 13, 2024
6853931
Fix handling error
ToukL Jun 13, 2024
e0e2158
Update video jump by frame manual articles
mzur Jun 14, 2024
ef0298e
Clean code and remove useless stopSeeking
ToukL Jun 18, 2024
f50678b
Merge branch 'master' into jump-frame-by-frame
ToukL Jun 18, 2024
d1b93bb
Merge branch 'jump-frame-by-frame' of https://github.com/ToukL/biigle…
ToukL Jun 18, 2024
e89878f
Fix merging issues
ToukL Jun 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 7 additions & 17 deletions resources/assets/js/videos/components/videoScreen.vue
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,20 @@
></control-button>
</div>
<div class="btn-group">
<control-button
v-if="enableJumpByFrame"
:disabled="seeking"
icon="fa-caret-square-left"
title="Previous frame 𝗟𝗲𝗳𝘁 𝗮𝗿𝗿𝗼𝘄"
v-on:click="emitPreviousFrame"
></control-button>
<control-button
v-if="jumpStep!=0"
:disabled="seeking"
icon="fa-backward"
:title="jumpBackwardMessage"
@click="jumpBackward"
></control-button>
<control-button
v-if="enableJumpByFrame"
:disabled="seeking"
icon="fa-caret-square-left"
title="Previous frame 𝗟𝗲𝗳𝘁 𝗮𝗿𝗿𝗼𝘄"
v-on:click="emitPreviousFrame"
></control-button>
<control-button
v-if="playing"
icon="fa-pause"
Expand Down Expand Up @@ -555,16 +555,6 @@ export default {
this.setPaused(true);
this.resetInteractionMode();
},
emitPreviousFrame() {
this.$emit('seek', this.video.currentTime, true);
if(!this.seeking)
this.showPreviousFrame();
},
emitNextFrame() {
this.$emit('seek', this.video.currentTime, true);
if(!this.seeking)
this.showNextFrame();
},
adaptKeyboardShortcuts() {
if(this.enableJumpByFrame) {
Keyboard.off('ArrowRight', this.emitNext, 0, this.listenerSet);
Expand Down
80 changes: 55 additions & 25 deletions resources/assets/js/videos/components/videoScreen/videoPlayback.vue
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ export default {
// Allow a maximum of 100x magnification. More cannot be represented in the
// URL parameters.
minResolution: 0.01,
// parameter tracking seeking state specific for frame jump, needed because looking for seeking directly leads to error
seekingFrame: this.seeking,
mzur marked this conversation as resolved.
Show resolved Hide resolved
};
},
methods: {
Expand Down Expand Up @@ -153,7 +155,23 @@ export default {
handleSeeked() {
this.renderVideo(true);
},
// 3 next methods are a workaround to get previous and next frames, adapted from here: https://github.com/angrycoding/requestVideoFrameCallback-prev-next/tree/main
// 5 next methods are a workaround to get previous and next frames, adapted from here: https://github.com/angrycoding/requestVideoFrameCallback-prev-next/tree/main
async emitPreviousFrame() {
if(this.video.currentTime == 0 || this.seekingFrame) return;
mzur marked this conversation as resolved.
Show resolved Hide resolved
this.$emit('start-seeking');
this.seekingFrame = true;
await this.showPreviousFrame();
this.$emit('stop-seeking');
this.seekingFrame = false;
},
async emitNextFrame() {
if(this.video.duration - this.video.currentTime == 0 || this.seekingFrame) return;
this.$emit('start-seeking');
this.seekingFrame = true;
await this.showNextFrame();
this.$emit('stop-seeking');
this.seekingFrame = false;
},
frameInfoCallback() {
let promise = new Vue.Promise((resolve) => {
this.video.requestVideoFrameCallback((now, metadata) => {
Expand All @@ -163,34 +181,46 @@ export default {
return promise;
},
async showPreviousFrame() {
// force rerender
this.video.currentTime += 1;
this.video.currentTime -= 1;
try {
// force rerender adapting step on begining or end of video
let step = 1;
if(this.video.currentTime < 1) {step = this.video.currentTime;}
if(this.video.duration - this.video.currentTime < 1) {step = this.video.duration - this.video.currentTime;}
mzur marked this conversation as resolved.
Show resolved Hide resolved
await (this.video.currentTime += step);
await (this.video.currentTime -= step);
mzur marked this conversation as resolved.
Show resolved Hide resolved

// get current frame time
const firstMetadata = await this.frameInfoCallback();

for (;;) {
// now adjust video's current time until actual frame time changes
this.video.currentTime -= 0.01;
let metadata = await this.frameInfoCallback();
if (metadata.mediaTime !== firstMetadata.mediaTime) break;
}
// get current frame time
const firstMetadata = await this.frameInfoCallback();
for (;;) {
// now adjust video's current time until actual frame time changes
this.video.currentTime -= 0.01;
// check that we are not at first frame, otherwise we'll end up in infinte loop
if (this.video.currentTime == 0) break;
const metadata = await this.frameInfoCallback();
if (metadata.mediaTime !== firstMetadata.mediaTime) break;
}
} catch(e) {console.error(e)}
},
async showNextFrame() {
// force rerender
this.video.currentTime += 1;
this.video.currentTime -= 1;

// get current frame time
const firstMetadata = await this.frameInfoCallback();
try {
// force rerender adapting step on begining or end of video
let step = 1;
if(this.video.currentTime < 1) {step = this.video.currentTime;}
if(this.video.duration - this.video.currentTime < 1) {step = this.video.duration - this.video.currentTime;}
await (this.video.currentTime += step);
await (this.video.currentTime -= step);

for (;;) {
// now adjust video's current time until actual frame time changes
this.video.currentTime += 0.01;
let metadata = await this.frameInfoCallback();
if (metadata.mediaTime !== firstMetadata.mediaTime) break;
}
// get current frame time
const firstMetadata = await this.frameInfoCallback();
for (;;) {
// now adjust video's current time until actual frame time changes
this.video.currentTime += 0.01;
// check that we are not at last frame, otherwise we'll end up in infinte loop
if (this.video.duration - this.video.currentTime == 0) break;
const metadata = await this.frameInfoCallback();
if (metadata.mediaTime !== firstMetadata.mediaTime) break;
}
} catch(e) {console.error(e)}
},
// Methods to jump back and forward in video. Step is given by parameter jumpStep.
jumpBackward() {
Expand Down
22 changes: 22 additions & 0 deletions resources/assets/js/videos/videoContainer.vue
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,28 @@ export default {

return promise;
},
startSeeking() {
if (this.seeking) {
return Vue.Promise.resolve();
}
let promise = new Vue.Promise((resolve, reject) => {
this.video.addEventListener('seeking', resolve);
this.video.addEventListener('error', reject);
});
this.seeking = true;
return promise;
},
stopSeeking() {
if (!this.seeking) {
return Vue.Promise.resolve();
}
let promise = new Vue.Promise((resolve, reject) => {
this.video.addEventListener('seeked', resolve);
this.video.addEventListener('error', reject);
});
this.seeking = false;
return promise;
},
mzur marked this conversation as resolved.
Show resolved Hide resolved
selectAnnotation(annotation, time, shift) {
if (this.attachingLabel) {
this.attachAnnotationLabel(annotation);
Expand Down
4 changes: 2 additions & 2 deletions resources/views/manual/tutorials/videos/shortcuts.blade.php
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
</tbody>
<tbody>
<tr>
<td style="text-align:center">If <a href="{{route('manual-tutorials', ['videos', 'sidebar'])}}#jump-by-frame">Jump by frame</a> is disabled</td>
<td colspan="2">If <a href="{{route('manual-tutorials', ['videos', 'sidebar'])}}#jump-by-frame">Jump by frame</a> is disabled</td>
</tr>
</tbody>
<tbody>
Expand All @@ -32,7 +32,7 @@
</tbody>
<tbody>
<tr>
<td style="text-align:center">If <a href="{{route('manual-tutorials', ['videos', 'sidebar'])}}#jump-by-frame">Jump by frame</a> is enabled</td>
<td colspan="2">If <a href="{{route('manual-tutorials', ['videos', 'sidebar'])}}#jump-by-frame">Jump by frame</a> is enabled</td>
</tr>
</tbody>
<tbody>
Expand Down
2 changes: 1 addition & 1 deletion resources/views/manual/tutorials/videos/sidebar.blade.php
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@

<p>
<a name="jump-by-frame"></a> The jump by frame switch allows you to navigate frame by frame (forward and backward) in the video. Note that this is an experimental feature as it is only available in Chrome and may not always give the right frame, so please use it with caution.
<br>When the feature is enabled, new buttons will appear in the navigation bar and keyboard shortcuts are modified: use <button class="btn btn-default btn-xs"><i class="fa fa-arrow-left"></i></i></button> / <button class="btn btn-default btn-xs"><i class="fa fa-arrow-right"></i></i></button> to switch frames and 𝗦𝗵𝗶𝗳𝘁 + <button class="btn btn-default btn-xs"><i class="fa fa-arrow-left"></i></i></button> / <button class="btn btn-default btn-xs"><i class="fa fa-arrow-right"></i></i></button> to switch videos. When the feature is disabled, use old shortcuts to switch videos.
<br>When the feature is enabled, new buttons will appear in the navigation bar and keyboard shortcuts are modified: use <kbd>Arrow left</kbd> / <kbd>Arrow right</kbd> to switch frames and <kbd>Shift</kbd> + <kbd>Arrow left</kbd> / <kbd>Arrow right</kbd> to switch videos. When the feature is disabled, use old shortcuts to switch videos.
</p>

</div>
Expand Down
2 changes: 2 additions & 0 deletions resources/views/videos/show/content.blade.php
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@
v-on:attaching-active="handleAttachingLabelActive"
v-on:swapping-active="handleSwappingLabelActive"
v-on:seek="seek"
v-on:start-seeking="startSeeking"
v-on:stop-seeking="stopSeeking"
v-on:is-invalid-shape="handleInvalidShape"
></video-screen>
<video-timeline
Expand Down