Skip to content

Latest commit

 

History

History
156 lines (116 loc) · 4.71 KB

README.md

File metadata and controls

156 lines (116 loc) · 4.71 KB
Raw

BackgroundMattingV2

English | 简体中文

  • BackgroundMattingV2
    • [1 Introduction](#1 Introduction)
    • [2 Accuracy](#2 Accuracy)
    • [3 Dataset](#3 Dataset)
    • [4 Environment](#4 Environment)
    • [5 Quick start](#5 Quick start)
    • [6 Code structure](#6 Code structure)
      • 6.1 structure
      • [6.2 Parameter description](#62-Parameter description)
      • [6.3 Training process](#63-Training process)
    • [7 Model information](#7 Model information)

1 Introduction

This project reproduces BackgroundMattingV2 based on paddlepaddle framework. BackgroundMattingV2 is divided into two parts: the base and the refine part. The base part generates a rough result output with a low resolution input and is used to provide a coarse regional location.Based on this, the refine network selects a fixed number of PATHS (these areas tend to select hair/hands and other difficult-to-distinguish areas) through path selection for refine. After that, the updated results of path are filled back to the original results to obtain their matting results in high resolution. image

Paper:

  • [1] Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian Curless, Steve Seitz, and Ira Kemelmacher Shlizerman. Real-time high-resolution background matting. In Computer Vision and Pattern Regognition (CVPR), 2021.

Reference project：

The link of aistudio：

2 Accuracy

Accuracy：SAD: 7.58，MSE: 9.49

This index is tested in the test set of PhotoMatte85

epoch opt learning_rate pretrain dataset SAD MSE
stage1 1 Adam 1e-4 none VideoMatte240K 11.68 12.85
stage2 300 Adam 5e-5 stage1.model(step_109999) Distinctions646_person 7.58 9.49
stage3 300 Adam 3e-5 stage2.model(epoch_169) private 7.61 9.47

Model Download Address： https://pan.baidu.com/s/1WfpzLcjaDJPXYSrzPWvsyQ code：nsfy

3 Dataset

VideoMatte240K & PhotoMatte85 dataset

  • Dataset size：
    • train：237,989
    • val：2,720
    • test：85

Distinctions646_person dataset

  • Dataset size：
    • train：362
    • val：11

4 Environment

  • Hardware: GPU, CPU

  • Framework:

    • PaddlePaddle >= 2.1.2

5 Quick start

step1: clone

# clone this repo
git clone https://github.com/PaddlePaddle/Contrib.git
cd BackgroundMattingV2
export PYTHONPATH=./

step2: train

sh ./run.sh

Because it is a target segmentation task, we need to pay attention to the gradual decrease of loss and the gradual decrease of SAD、MSE.

step3: test

python3 eval.py

According to the test set designed in the original paper, the data will be randomly augmented, so the results will fluctuate.

Prediction using pre training model

python3 predict.py

save the output image in the ./image

6 Code structure

6.1 structure

├─dataset                        
├─image                        
├─log                          
├─model                       
├─utils                          
│  eval.py                   
│  predict.py                
│  README.md                   
│  README_cn.md                 
│  run.sh                      
│  train.py

6.2 Parameter description

Parameters related to training and evaluation can be set in train.py, as follows:

Parameters description
--dataset-name Name of datasets
--learning-rate Learning rate
--log-train-loss-interval Print the step of loss
--epoch_end Num of epoch
--pretrain Parameter path of pre training model

6.3 Training process

Single machine training

sh ./run.sh

7 Model information

For other information about the model, please refer to the following table:

information description
Author Jialei Zhao
Date 2021.10
Framework version Paddle 2.1.2
Application scenarios High resolution matting
Support hardware GPU、CPU
Download link Pre training model code：6fnd
Online operation botebook