is there a way to modify model weights based on timestamps?

[afsara-ben]

is there a way to get timestamp based logits? such as for a timestamp 0.4-0.6 is it possible to find the model weights/logits produced in this timespan? i understand the logits are on token-level (i.e for each token, there is a logit of shape (1,51685)). is there a way to modify model weights based on timestamps? such as i want to modify the weights that correspond to timestamp 0.4-0.6.

[Jeronymous]

Timestamps corresponding to start/end of segments (not words) are special tokens with probability, that could be included in the output transcription (for segments, not words).
Like the likelihood of the Whisper predictions for start/end segment.
They could be accumulated over a span of timestamps (around the real prediction)?

Not sure what is exactly the requested feature here.
Do you want to access an information?
Or to have an influence on the model output? (I am not getting "such as i want to modify the weights that correspond to timestamp 0.4-0.6")

[afsara-ben]

my understanding is that the model output logits are per token. and we know the timestamp around each/group of tokens. what i am asking is, is there a way to get the logits that are associated with a certain timespan? so if timespan 0.4-0.6 has tokens [312, 45, 856], is there a direct way to get the logits that would belong to that timespan? i would like to update the model weights based on this timestamps -- hypothetically if 0.4-0.6 has a label 1, the model will be trained to predict that label.