convert_graph_to_onnx.py failing to run on Wav2Vec2 models

[hrahamim]

Environment info

• transformers version: 4.6.1
• Platform: Linux-4.15.0-143-generic-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.9
• PyTorch version (GPU?): not installed (NA)
• Tensorflow version (GPU?): not installed (NA)
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Information

Model I am using (Bert, XLNet ...): facebook/wav2vec2-large-960h

The problem arises when using:

• [X ] the official example scripts: (give details below)
  I am trying the next command:
  python transformers/src/transformers/convert_graph_to_onnx.py --framework pt --quantize --model facebook/wav2vec2-large-960h wav2vec2_convert.onnx

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

run:
python transformers/src/transformers/convert_graph_to_onnx.py --framework pt --quantize --model facebook/wav2vec2-large-960h wav2vec2_convert.onnx

I get the next error:

python transformers/src/transformers/convert_graph_to_onnx.py --framework pt --quantize --model facebook/wav2vec2-large-960h wav2vec2_convert.onnx
/home/ptuser/anaconda3/envs/gong_env/lib/python3.6/site-packages/torchaudio/backend/utils.py:54: UserWarning: "sox" backend is being deprecated. The default backend will be changed to "sox_io" backend in 0.8.0 and "sox" backend will be removed in 0.9.0. Please migrate to "sox_io" backend. Please refer to pytorch/audio#903 for the detail.
'"sox" backend is being deprecated. '

====== Converting model to ONNX ======
ONNX opset version set to: 11
Loading pipeline (model: facebook/wav2vec2-large-960h, tokenizer: facebook/wav2vec2-large-960h)
Some weights of the model checkpoint at facebook/wav2vec2-large-960h were not used when initializing Wav2Vec2Model: ['lm_head.bias', 'lm_head.weight']

• This IS expected if you are initializing Wav2Vec2Model from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing Wav2Vec2Model from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Some weights of Wav2Vec2Model were not initialized from the model checkpoint at facebook/wav2vec2-large-960h and are newly initialized: ['wav2vec2.masked_spec_embed']
  You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
  Error while converting the model: init() got an unexpected keyword argument 'feature_extractor'

Expected behavior

[LysandreJik]

Hi, Wav2Vec2 is not supported by the ONNX converter. Please check out the following PR which offers the possibility to convert existing models to ONNX by defining an architecture: #11786. You can see the docs (WIP) for that PR here.

[hrahamim]

Will be happy if Wav2Vec2 will be supported in the future...

[patrickvonplaten]

Think someone solved it here: #10004

[hrahamim]

Thanks Patrick