You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+6-4
Original file line number
Diff line number
Diff line change
@@ -1,9 +1,9 @@
1
1
# Colab Implement: ConVIRT model
2
2
### Contrastive VIsual Representation Learning from Text
3
3
4
-
The repo is a Colab implementation of the architecture descibed in the ConVIRT paper: [*Contrastive Learning of Medical Visual Representations from Paired Images and Text*](https://arxiv.org/abs/2010.00747). The authors of paper are Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D. Manning, Curtis P. Langlotz.
4
+
Deep neural networks learn from a large amount of data to obtain the correct parameters to perform a specific task. However, in practice, we often encounter a problem: **insufficient amount of labeled data**. However, if your data contains pairs of images and text, you can solve the problem with c.
5
5
6
-
Deep neural networks learn from a large amount of data to obtain the correct parameters to perform a specific task. However, in practice, we often encounter a problem: **insufficient amount of labeled data**. However, if your data contains pairs of imagesand text, you can solve the problem with Contrastive Learning.
6
+
Contrastive learning is a kind of self-supervised learning method. It does not require specialized labels, but rather a method to learn the correct parameters from the unlabeled data itself. It aims to learn an encoder that makes the encoding results of similar classes of data similar and makes the encoding results of different classes of data as different as possible. Typical contrast learning is done based on comparisons between two images. However, if we have paired image and text data, contrast learning can also be applied between images and text.
7
7
8
8
Based on this repository, we can implement various paired-image-text Contrastive Learning tasks on [Google Colab](https://colab.research.google.com/notebooks/intro.ipynb), which enable you to train effective pre-training models for transfer learning with insufficient data volume. With this pre-trained model, you can train with less labeled data to get a good performing model.
9
9
@@ -61,7 +61,7 @@ loss:
61
61
alpha_weight: 0.75
62
62
```
63
63
64
-
The models used (res_base_model, bert_base_model) refers to the models provided by [transformers](https://huggingface.co/transformers/).
64
+
The models used [res_base_model, bert_base_model] refers to the models provided by [transformers](https://huggingface.co/transformers/).
65
65
66
66
### 3. Training
67
67
@@ -76,7 +76,9 @@ At the end of training, the final model and the corresponding config.yaml will b
76
76
77
77
## Others
78
78
79
-
Note: This repository was forked and modified from https://github.com/sthalles/SimCLR.
79
+
The repository is a Colab implementation of the architecture descibed in the ConVIRT paper: [*Contrastive Learning of Medical Visual Representations from Paired Images and Text*](https://arxiv.org/abs/2010.00747). The authors of paper are Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D. Manning, Curtis P. Langlotz.
80
+
81
+
This repository was originally modified from https://github.com/sthalles/SimCLR.
80
82
81
83
References:
82
84
- Yuhao Zhang et al. Contrastive Learning of Medical Visual Representations from Paired Images and Text. https://arxiv.org/pdf/2010.00747.pdf
0 commit comments