SmartFit

A virtual fitting room service for e-commerce!

• Important Note
• Requisites
• Setup
  • Test Example
• Notebooks
• References

Important Note

This repository is for demo purposes only and runs a publicly available clothing transfer model, which does not include all improvements. Correspondingly, some functionality may be missing.

Requisites

• Conda
• Octave 4.2+ (with image package)
  • Also compatible with MATLAB

Setup

First clone the repository:

git clone https://github.com/RohanBhandari/SmartFit.git
cd SmartFit

Next, run source setup.sh, which will create a conda environment and install the required packages via

conda create --name smartfit python=2.7 pip
source activate smartfit
pip install -r requirements.txt

Additionally, setup.sh will clone and setup auxiliary repositories that do the human parsing, pose estimation, and clothing transfer steps. This process will download pre-trained models for each of the tasks.

Test Example

To test whether the setup was successful, run the following command that should reproduce the clothing transfer shown at the top of the README:

./run_smartfit.sh inputs/example_person.jpg inputs/example_clothing.jpg

The corresponding output is output/output.png and should be similar to output/example_output.png (the right-most image above).

Notebooks

This repo includes some jupyter notebooks (located in notebooks/) that may be useful for understanding the SmartFit pipeline and its inputs/outputs. Exact usage instructions are at the top of each notebook.

• visualize_inputs.ipynb
  • Explains and shows the main inputs to the clothing transfer model: body mask, face and pants, keypoint pose map, and clothing item.
• visualize_output.ipynb
  • Displays the output image.
• skintone_check.ipynb
  • Describes how the skintone check works. This is a check to make sure that the clothing transfer model did not change the skintone of the inputted person.

References

This project builds from the work listed below:

• Human parsing
  • LIP_JPPNet (repo, paper)
• Pose estimation
  • Realtime Multi-Person Pose Estimation (repo, paper)
    • Keras implementation
• Virtual try-on
  • VITON (repo, paper)