Adds HuggingFace pretrained encoder unit tests

[geoffreyangus]

No description provided.

[github-actions]

Unit Test Results

         6 files  +    1           6 suites  +1   5h 13m 15s ⏱️ + 1h 4m 17s
  3 933 tests +  31    3 877 ✔️ +  49    56 💤  - 18  0 ❌ ±0 
11 796 runs  +283  11 628 ✔️ +314  168 💤  - 31  0 ❌ ±0 

Results for commit 8b8e44f. ± Comparison against base commit 26b36c3.

♻️ This comment has been updated with latest results.

[arnavgarg1]

Can you considering adding this one as well?

@pytest.mark.parametrize(
    "encoder_config_cls, encoder_cls",
    [
        (text_encoders_configs.ALBERTConfig, text_encoders.ALBERTEncoder),
        (text_encoders_configs.BERTConfig, text_encoders.BERTEncoder),
        (text_encoders_configs.XLMConfig, text_encoders.XLMEncoder),
        (text_encoders_configs.GPTConfig, text_encoders.GPTEncoder),
        (text_encoders_configs.RoBERTaConfig, text_encoders.RoBERTaEncoder),
        (text_encoders_configs.GPT2Config, text_encoders.GPT2Encoder),
        (text_encoders_configs.DistilBERTConfig, text_encoders.DistilBERTEncoder),
        (text_encoders_configs.TransformerXLConfig, text_encoders.TransformerXLEncoder),
        (text_encoders_configs.CTRLConfig, text_encoders.CTRLEncoder),
        (text_encoders_configs.CamemBERTConfig, text_encoders.CamemBERTEncoder),
        (text_encoders_configs.MT5Config, text_encoders.MT5Encoder),
        (text_encoders_configs.XLMRoBERTaConfig, text_encoders.XLMRoBERTaEncoder),
        (text_encoders_configs.LongformerConfig, text_encoders.LongformerEncoder),
        (text_encoders_configs.ELECTRAConfig, text_encoders.ELECTRAEncoder),
        (text_encoders_configs.FlauBERTConfig, text_encoders.FlauBERTEncoder),
        (text_encoders_configs.T5Config, text_encoders.T5Encoder),
        (text_encoders_configs.XLNetConfig, text_encoders.XLNetEncoder),
        (text_encoders_configs.DistilBERTConfig, text_encoders.DistilBERTEncoder),
    ],
)
def test_hf_default_reduce_output_strategy(
    encoder_config_cls: text_encoders_configs.BaseEncoderConfig, encoder_cls: text_encoders.HFTextEncoder
):
    """Test that default reduce_output strategy works for all text HFencoders."""
    reduce_output_strategy = encoder_config_cls.reduce_output
    encoder = encoder_cls(use_pretrained=False, reduce_output=reduce_output_strategy, max_sequence_length=20)
    inputs = torch.rand((2, 20)).type(encoder.input_dtype)
    encoder(inputs)

It may need some tweaking

[arnavgarg1]

🚢