Discourse probing of transformer-based models

This repo contains experiment outputs and source code related to Master's thesis on discourse probing of transformer-based models by Vadim Fomin. A paper describing these experiments is currently in progress.

Embeddings extractiion

Scripts in the directory embeddings_extraction contain code that was used for extracting embeddings of texts from BERT, RuBERT, and GPT-2.

Diagnostic classification

Files diagnostic_classification.py and diagnostic_classification.py contain the code that was used for performing experiments over the extracted embeddings.