Wav2Vec2

[ChrisBeeson]

How tricky would it be to implement Wav2Vec2?

[guillaume-be]

Hello @ChrisBeeson ,

The transformer model itself is probably rather straightforward to implement as it seems to use standard Torch operations throughout that are supported in tch-rs.

The tricky part is the pre-processing and tokenization process for the model that would differ significantly from the existing tokenizer, since the input would be an audio file. One would need to identify an alternative in Rust to load audio files, build a sampling, tensor conversion and padding module and the associated tokenizer, which would represent a significant amount of work.

I would welcome a pull request for this feature, but this would probably meaning getting started with a pre-processing/tokenization implementation, either directly in https://github.com/guillaume-be/rust-tokenizers or as in a separate crate/dependency.

[ChrisBeeson]

Hi @guillaume-be,

Thanks for your thoughts on this.

Would porting the hugging face tokenizer and the feature extractor be a good starting point?

[j4qfrost]

I've started some work here guillaume-be/rust-tokenizers#90

Can I get some input?