OLD Template for ML models

THE CONTENT IN THIS SECTION IS VALID UNTIL v4.4, WHERE THESE MACHINE LEARNING MODELS WERE UPDATED IN FAVOUR OF MORE PRECISE ONES.

The model management has been inspired by spaCy models

Structure

Example of the structure of the path model

path_model-1.0.0
 |- README.md
 |- MANIFEST.in
 |- meta.json
 |- setup.py
 `- path_model
     |- __init__.py
     |- meta.json
     `- path_model-1.0.0
         `- path_model.bin

Metafile

meta.json

{
    "name": "path_model",
    "version": "1.0.0",
    "credentialdigger_version": ">=1.0.0",
    "parent_package":"credentialdigger",
    "description": "Description of the model here",
    "author": "Author name",
    "email": "contact@example.com",
    "url": "https://example.com",
    "license": "Apache 2"
}

How to create a new model

1. Copy the structure shown above, using appropriate name and version for the model
2. Update the metafile meta.json
   • name: name of the model (e.g., path_model)
   • version: version of the model (e.g., 1.0.0)
   • credentialdigger_version: versions of credentialdigger supported. The minimum should be at least 1.0.0
   • parent_package: credentialdigger
   • description: a description of the model
   • author, email, url: information on author and/or maintainer
   • license: the license
3. Make sure to use the semantic name-version where needed, and note that both name and version must correspond to the values of the metafile
4. In the inner folder, drop in the binary file(s)

How to share the new model

Follow this template as guidelines to create a model for credentialdigger. Refer to the previous section for a HowTo.

cd ..
tar -czvf model_name-x.x.x.tar.gz credentialdigger_model_template

Example:

cd .. && tar -czvf path_model-1.0.0.tar.gz personal_model-1.0.0

Publish the file .tar.gz somewhere. Save the link in an environment variable called same as the model

export modelname=https://........

Example:

export personal_model=https://127.0.0.1:5000/models/personal_model-1.0.0.tar.gz

Download and install the model (don't run this command from the credentialdigger installation folder), unless you installed credentialdigger via pip

python -m credentialdigger download modelname

Example:

python -m credentialdigger download personal_model

Example

A pre-trained File Path model is available here.

It has been generated as illustrated above, and it can be installed as follows:

export path_model=https://github.com/SAP/credential-digger/releases/download/v1.0.0/path_model-1.0.0.tar.gz
python -m credentialdigger download path_model