progress tracking, tables, relationships, and API

[JessieLOliver]

Mechanisms to assign and track processing of audio segments are essential:
1. for users and researchers to know what data has already been reviewed
2. to assign audio clips for sampling in a strategic manner
3. to allow for large-scale visualization of data
4. to test for consistency/accuracy across users for species identification
5. Allow participants to identify easily when target species have been found (and for verification)

---

Formal specification

Tracking progress for activities on the website. Main use case, tracking amount of audio listened to by user. This functionality will eventually help our long term goals of tracking work assignments.

Suggested table design:

• progress(user_id, recording_id, activity, start_offset, end_offset, created_at, modified_at)
• or progress(user_id, recording_id, activity, offsets:jsonb, created_at, modified_at)

That second table schema makes use of postgres' jsonb functionality to store offset data. It should drastically reduce the number of rows that need to be stored. Additionally, we may be able to store cached or extra values in the JSON document.

Whichever design is chosen, there will need to be a regular compact and sweep operation that reduces the number of offsets stored. For example, with offset pairs (start, end): (120,180), (180,210), (210,240), (500,530) the compact operation should result in: (120,240), (500,530. This algorithm should bridge short time spans; initial estimation of bridging variable is 1.0 seconds.

The chosen schema will need to be support queries like:

• number of hours spent listening on user profile
• number of hours listened in site wide statistics

The bulk of this issue will be implementing table and standard API. Suggested endpoint: /progress (short form) and /audio_recordings/:audio_recording.id/progress.

However as a proof of concept we should integrate some basic stats into the user's profile.

Optional extensions

These optional extensions are potentially useful but a cost benefit discussion is needed.

• Supporting count times a segment has been (i.e. listened) to
• Recording dates that segments have been listened to (enable tracking work progress over time)

Both of these features would have very large performance impacts and would greatly affect the efficiency of the sweep and compact algorithm.

Cost and planning

Note: This is a very low priority feature. Work should not start until it has been approved.

This feature could be very complex to implement.

---

Related Issues

• baw-client: progress tracking, using API [https://github.com/progress tracking, using API  baw-client#263]
• baw-client: progress tracking, visualization
• Ecoacoustics: Long term goal: work assignment
• Ecoacoustics: Long term goal: sampling work assignment

[JessieLOliver]

@atruskie if you could add more technical detail to this and related issues that would be most helpful.

[atruskie]

Feedback added.

[atruskie]

This idea has been superseded by #348 - but good news! It is actually on its way to being implemented.