Skip to content

Latest commit

 

History

History
73 lines (69 loc) · 3.38 KB

README.md

File metadata and controls

73 lines (69 loc) · 3.38 KB

frankmocap_notebooks

  • This repository is to use the FrankMocap easier.. (MAYBE)
  • Disclaimer: This repo. would be outdated since FrankMocap is updated and we forgot the fork the previous version. Please visit "youtube_frankmocap" instead.

Preparation

  1. Prepare Ubuntu 18.04+ and Anaconda
  2. Clone this Repository and FrankMocap
git clone https://github.com/cotton-ahn/frankmocap_notebooks
cd frankmocap_notebooks
git clone https://github.com/facebookresearch/frankmocap
  1. Install the FrankMocap as the official document says
  • When installing, you may face error related to "opendr" installation
  • If ERRROR MSG goes like
    • Collect2: error: ld returned 1 exit status // Error: command ‘gcc’ failed with exit status 1
    • TRY below line : source
    sudo apt-get install libosmesa6-dev
  1. Try to run below to check whether the result is valid.
cd ./frankmocap
conda activate venv_frankmocap
python -m demo.demo_bodymocap --input_path ./sample_data/han_short.mp4 --out_dir ./mocap_output
  1. Replace ./copy_and_paste.py and ./demo_frankmocap.py
cd $(THIS REPOSITORY)
cp ./copy_and_paste.py ./frankmocap/integration
cp ./demo_frankmocap.py ./frankmocap/demo
  1. Prepare the jupyter notebook, and add the venv_frankmocap to the ipykernel
pip install ipykernel jupyter
python -m ipykernel install --user --name=venv_frankmocap
  1. Install moviepy
pip install moviepy
  1. Run Jupyter Notebook in this repository
cd $(THIS REPOSITORY)
jupyter notebook

How to Use

  • Put videos that you want to process, to ./videos (config.video_dir)
    • Or, you can write down the YouTube Links on ./video_url.txt
    • Then, Run 00_process_youtube_videos.ipynb
    • Videos will be saved to ./videos, image frames will be saved to ./images (config.image_dir)
  • Run 01_extract_mocap_results.
    • Results will be saved to ./mocaps (config.mocap_dir)
      • If config.read_type == 'videos', video files in ./videos will be used as inputs to the frankmocap
      • If config.read_type == 'images', images in ./images will be used as inputs to the frankmocap
    • In ./mocaps, folders with video's name will be generated, and results will be saved.
  • Run 02_preprocess_mocap_result
    • This notebook organizes results into some .pkl and .npz files.
    • .pkl files will be saved to ./skeletons_pkl (config.skeletons_pkl_dir)
    • .npz files will be saved to ./skeletons_npz (config.skeletons_npz_dir)
  • See the result with 03_visualize_processed_pkl_and_npz.
    • After loading pickle file, you will see the dictionary
      • KEY : 'open_pose', 'aux_pose', 'simple_pose', 'left_hand', 'right_hand', 'hand_info', 'heel_info', 'head_info'
      • Try to check the pickle file by yourself.
    • After loading .npz file, you will see an 87-dimensional array.
      • 0 ~ 45 (45) : 'simple_pose' in pickle file, the pose which size was 15 by 3
      • 45~ 51 (6) : (3) head pose (3) head orientation
      • 51 ~ 63 (12): left hand thumb vector(3), index vector(3), palm vector(3), wrist pos(3)
      • 63 ~ 75 (12): right hand thumb vector(3), index vector(3), palm vector(3), wrist pos(3)
      • 75 ~ 87 (12): left heel pos (3), left foot vec (3), right heel pos (3), right foot vec (3)