Improving Multi-Modal E-Commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization
- Python 3.6.8
- CUDA 10.1
- cudnn 7.6.2
- PyTorch 1.6.0
- TorchVision 0.7.0
- NumPy 1.19.5
- pandas 1.1.5
- Pillow 8.4.0
- Install the non-Python dependences including Nvidia GPU Driver, CUDA and Python. And 1x Nvidia V100 GPU (32G Version) is required for evaluation.
- Run
pip3 install -r requirementsfor install Python dependences.
- Download the datasets:
-
MEPAVE: Following the instruction of official repository https://github.com/jd-aig/JAVE to fillup the application form, and then E-mail it to the contact author for the accessment of this dataset.
-
MAE: Download the annotations of train and val subsets on official website https://rloganiv.github.io/mae/. Due to the zipped image files are corrupted, we have contacted the authors and obtained the new download link: https://drive.google.com/file/d/1g5a2NiKreNJx5fR1__8HraBoDkPQXR6Z/view?usp=sharing
-
Place the unarchived dataset files on path wherever you like, and modify the global paths in
paths.py. For the tokenization of input texts during inference time, pretrained BERT models are needed, please refer to the links inpaths.pyto download and modify the corresponding paths. (p.s. ResNet is only used for joinly fine-tuning during training) -
For MAE dataset, an extra pre-processing is needed to convert the annotation from attribure-value pairs to "B-I-O" sequences. Run
python3 convert_mae.pyfor this conversion.
-
Run
python3 test_mae.pyto obtain the evaluation results of ULS-DRAM on MAE benckmark. -
Run
python3 test_mepave.pyto obtain the evaluation results of ULS-DRAM on MEPAVE benckmark.
You will get the final output as follows:
MAE:
CLS eval results: Precision, Recall, F1
0.83447 0.80507 0.81951
TAG eval results: Precision, Recall, F1
0.85429 0.78038 0.81567
Accuracy score: 79.20237
MEPAVE:
CLS eval results: Precision, Recall, F1
0.97285 0.98024 0.97653
TAG eval results: Precision, Recall, F1
0.95887 0.97862 0.96864