Skip to content

Commit

Permalink
Align model card requirements (nv-morpheus#1388)
Browse files Browse the repository at this point in the history 
Authors:
  - Devin Robison (https://github.com/drobison00)

Approvers:
  - David Gardner (https://github.com/dagardner-nv)
  - Michael Demoret (https://github.com/mdemoret-nv)

URL: nv-morpheus#1388
  • Loading branch information
@drobison00
drobison00 committed Nov 27, 2023
1 parent 119ec42 commit 5f00e78
Show file tree
Hide file tree
Showing 5 changed files with 13 additions and 107 deletions.
20 changes: 0 additions & 20 deletions models/model-cards/abp-model-card.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,6 @@ limitations under the License.

* Sample dataset consists of over 1000 nvidia-smi outputs <br>

**Dataset License:**

* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) <br>

## Evaluation Dataset:

**Link:**
Expand All @@ -108,10 +104,6 @@ limitations under the License.

* Sample dataset consists of over 1000 nvidia-smi outputs <br>

**Dataset License:**

* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)<br>

## Inference:

**Engine:**
Expand Down Expand Up @@ -213,10 +205,6 @@ limitations under the License.

* N/A

### What training is recommended for developers working with this model?

* Familiarity with the Morpheus SDK is recommended for developers working with this model.

### Link the relevant end user license agreement

* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)
Expand Down Expand Up @@ -256,10 +244,6 @@ limitations under the License.

* N/A

### Technical robustness and model security validated?

* No

### Is the model and dataset compliant with National Classification Management Society (NCMS)?

* No
Expand Down Expand Up @@ -308,10 +292,6 @@ limitations under the License.

* N/A

### Scanned for malware?

* No

### Are we able to identify and trace source of dataset?

* Yes
Expand Down
15 changes: 0 additions & 15 deletions models/model-cards/dfp-model-card.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,9 +83,6 @@ The training dataset consists of AWS CloudTrail logs. It contains logs from two
* [hammah-user123-training-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/models/datasets/training-data/cloudtrail/hammah-user123-training-part3.json): 1000 records <br>
* [hammah-user123-training-part4.json](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/models/datasets/training-data/cloudtrail/hammah-user123-training-part4.json): 387 records <br>

**Dataset License:**
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)<br>

## Evaluation Dataset:
**Link:**
* https://github.com/nv-morpheus/Morpheus/tree/branch-23.11/models/datasets/validation-data/cloudtrail <br>
Expand All @@ -98,9 +95,6 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
* [hammah-user123-validation-part2.json](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part2.json): 300 records
* [hammah-user123-validation-part3.json](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/models/datasets/validation-data/cloudtrail/hammah-user123-validation-part3.json): 247 records

**Dataset License:**
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)<br>

## Inference:
**Engine:**
* PyTorch
Expand Down Expand Up @@ -179,9 +173,6 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
### What are the potential known risks to users and stakeholders?
* None

### What training is recommended for developers working with this model? If none, please state "none."
* Familiarity with the Morpheus SDK is recommended for developers working with this model.

### Link the relevant end user license agreement
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)

Expand Down Expand Up @@ -211,9 +202,6 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
### Name target quality Key Performance Indicators (KPIs) for which this has been tested.
* None

### Technical robustness and model security validated?
* No

### Is the model and dataset compliant with National Classification Management Society (NCMS)?
* No

Expand Down Expand Up @@ -251,9 +239,6 @@ The evaluation dataset consists of AWS CloudTrail logs. It contains logs from tw
### Is data in dataset traceable?
* No

### Scanned for malware?
* No

### Are we able to identify and trace source of dataset?
* Yes ([fully synthetic dataset](https://github.com/nv-morpheus/Morpheus/tree/branch-23.11/models/datasets/training-data/cloudtrail))

Expand Down
27 changes: 13 additions & 14 deletions models/model-cards/gnn-fsi-model-card.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,13 @@ This model is an example of a fraud detection pipeline using a graph neural netw
**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* A training data consists of raw 753 synthetic labeled credit card transaction data with data augmentation in a total of 12053 labeled transaction data. <br>

**Dataset License:**
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0) <br>

## Evaluation Dataset:
**Link:**
* [fraud-detection-validation-data.csv](models/dataset/fraud-detection-validation-data.csv) <br>

**Properties (Quantity, Dataset Descriptions, Sensor(s)):**
* Data consists of raw 265 labeled credit card transaction synthetically created<br>

**Dataset License:**
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)<br>

## Inference:
**Engine:**
* Triton <br>
Expand Down Expand Up @@ -124,16 +118,21 @@ This model is an example of a fraud detection pipeline using a graph neural netw

### What is the accent balance of the model validation data?
* Not Applicable

### Describe measures taken to mitigate against unwanted bias.
* Not Applicable

## Model Card ++ Explainability Subcard

### Name example applications and use cases for this model.
* The model is primarily designed for testing purposes and serves as a small pretrained model specifically used to evaluate and validate the GNN FSI pipeline. Its application is focused on assessing the effectiveness of the pipeline rather than being intended for broader use cases or specific applications beyond testing.

### Fill in the blank for the model technique.
* This model is designed for developers seeking to test the GNN fraud detection pipeline with a small pretrained model on a synthetic dataset.

### Name who is intended to benefit from this model.
* The intended beneficiaries of this model are developers who aim to test the performance and functionality of the GNN fraud detection pipeline using synthetic datasets. It may not be suitable or provide significant value for real-world transactions.

### Describe the model output.
* This model outputs fraud probability score b/n (0 & 1).

Expand All @@ -152,8 +151,6 @@ This model is an example of a fraud detection pipeline using a graph neural netw
### What are the potential known risks to users and stakeholders?
* None

### What training is recommended for developers working with this model? If none, please state "none."
* Familiarity with the Morpheus SDK is recommended for developers working with this model.
### Link the relevant end user license agreement
* [Apache 2.0](https://github.com/nv-morpheus/Morpheus/blob/branch-23.11/LICENSE)

Expand All @@ -170,17 +167,19 @@ This model is an example of a fraud detection pipeline using a graph neural netw

### Was model and dataset assessed for vulnerability for potential form of attack?
* No

### Name applications for the model.
* Used for testing fraud detection application in Morpheus pipeline, under the defined dataset schema description.

### Name use case restrictions for the model.
* The model's use case is restricted to testing the Morpheus pipeline and may not be suitable for other applications.

### Has this been verified to have met prescribed quality standards?
* No

### Name target quality Key Performance Indicators (KPIs) for which this has been tested.
* Not Applicable
### Technical robustness and model security validated?
* Not Applicable

### Is the model and dataset compliant with National Classification Management Society (NCMS)?
* Not Applicable

Expand All @@ -189,38 +188,38 @@ This model is an example of a fraud detection pipeline using a graph neural netw

### Are there access restrictions to systems, model, and data?
* No

### Is there a digital signature?
* No

## Model Card ++ Privacy Subcard

### Generatable or reverse engineerable personally-identifiable information (PII)?

* Neither

### Was consent obtained for any PII used?
* Not Applicable (Data is extracted from synthetically created credit card transaction,refer[3] for the source of data creation)

### Protected classes used to create this model? (The following were used in model the model's training:)

* Not applicable

### How often is dataset reviewed?
* The dataset is initially reviewed upon addition, and subsequent reviews are conducted as needed or upon request for any changes.

### Is a mechanism in place to honor data
* Yes

### If PII collected for the development of this AI model, was it minimized to only what was required?
* Not applicable

### Is data in dataset traceable?
* No
### Scanned for malware?
* No

### Are we able to identify and trace source of dataset?
* Yes

### Does data labeling (annotation, metadata) comply with privacy laws?
* Not applicable

### Is data compliant with data subject requests for data correction or removal, if such a request was made?
* Not applicable
17 changes: 0 additions & 17 deletions models/model-cards/phishing-model-card.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,6 @@ limitations under the License.

* Dataset consists of SMSs <br>

**Dataset License:**

* https://creativecommons.org/licenses/by/4.0/legalcode taken from https://archive.ics.uci.edu/dataset/228/sms+spam+collection <br>

## Evaluation Dataset:

**Link:**
Expand All @@ -110,10 +106,6 @@ limitations under the License.

* Dataset consists of SMSs <br>

**Dataset License:**

* https://creativecommons.org/licenses/by/4.0/legalcode taken from https://archive.ics.uci.edu/dataset/228/sms+spam+collection <br>

## Inference:

**Engine:**
Expand Down Expand Up @@ -207,9 +199,6 @@ limitations under the License.
### What are the potential known risks to users and stakeholders?
* N/A

### What training is recommended for developers working with this model?
* Familiarity with the Morpheus SDK is recommended for developers working with this model.

### Link the relevant end user license agreement
* [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)

Expand Down Expand Up @@ -240,9 +229,6 @@ limitations under the License.
### Name target quality Key Performance Indicators (KPIs) for which this has been tested.
* N/A

### Technical robustness and model security validated?
* No

### Is the model and dataset compliant with National Classification Management Society (NCMS)?
* No

Expand Down Expand Up @@ -281,9 +267,6 @@ limitations under the License.
### Is data in dataset traceable?
* N/A

### Scanned for malware?
* No

### Are we able to identify and trace source of dataset?
* N/A

Expand Down
Loading

0 comments on commit 5f00e78

Please sign in to comment.