Evaluate F-score

This is a modified version of the F-score evaluation of 3D meshes provided by Thanks and Temples.

For improved evaluation realism, this version does not crop or downsample the meshes.

Prerequisites

The library has been tested with the following dependencies, but should work with other versions as well.

1. Python 3.8.5
2. Open3D 0.9.0
3. Matplotlib 3.3.3

Installation

1. Clone the repository to your local directory:

   git clone https://github.com/tfy14esa/evaluate_3d_reconstruction_lib.git

2. If you don't want to provide the full path to the ground truth mesh when you call the evaluation script, you can specify the path to the folder containing the ground truth meshes in a hard coded way (not recommended). Open the file

   evaluate_3d_reconstruction/evaluate_3d_reconstruction/config.py

   and revise the path to where you store the ground truth meshes.
3. Then open the file

   evaluate_3d_reconstruction/evaluate_3d_reconstruction/evaluate_3d_reconstruction.py

   and revise the shebang at the top to point to the python executable of your virtual environment (this can be useful if you want to execute the evaluation script directly from the command line (see below)).
4. Activate your virtual environment
5. Enter the root folder of the library:

   cd evaluate_3d_reconstruction_lib

6. Install the library:

   pip install .

Usage

The main function of the library is

def run_evaluation(pred_ply, path_to_pred_ply, scene, distance_thresh=0.10, gt_translate_to_zero=False, pred_translate_to_zero=False):
   """Calculates the F-score from a predicted mesh to a reference mesh. Generates
    a directory and fills this with numerical and mesh results.

        Args:
            pred_ply (string): string object to denote the name of predicted mesh (as a .ply file)
            path_to_pred_ply (string): string object to denote the full path to the pred_ply file
            scene (string): string object to denote the scene name (a corresponding ground truth .ply file with the name "scene + .ply" needs to exist)
            distance_threshold (float):
            gt_translate_to_zero (bool): boolean describing whether to translate gt mesh to origin
            pred_translate_to_zero (bool): boolean describing whether to translate predicted mesh to origin
            icp_align (bool): align the recontructed mesh with the gt using ICP
            full_path_to_gt_ply (string): specify full path to ground truth mesh

        Returns:
            None
    """

The main function can be called in two principled ways:

1. As an executable directly from the command line as:

evaluate_3d_reconstruction.py pred_ply scene full_path_to_gt_ply

To achieve this, run

chmod +x evaluate_3d_reconstruction.py

and export the path to the script in your bashrc-file i.e. add similar to the following to your bashrc:

# Export path to my python evaluate 3d reconstruction script
export PATH="/cluster/project/cvl/esandstroem/src/late_fusion_3dconv/deps/evaluate_3d_reconstruction/evaluate_3d_reconstruction:$PATH"

2. As a normal function in other python scripts. To achieve this, simply import the function using

   from evaluate_3d_reconstruction import run_evaluation