Home

Pre-training has become an essential part of AI technology. TencentPretrain is a toolkit for pre-training on different modalities (e.g. text and vision) and fine-tuning on downstream task. TencentPretrain maintains model modularity and supports research extensibility. It facilitates the use of existing pre-training models, and provides interfaces for users to further extend upon. With TencentPretrain, we build a model zoo which contains pre-trained models of different properties. TencentPretrain inherits the open source toolkit UER (https://github.com/dbiir/UER-py/) and extends it to a multimodal pre-training framework.