Updated ML glossary #5503
Updated ML glossary #5503
Conversation
|
|
||
| The proportion of true results to total cases. Ranges from 0 (least accurate) to 1 (most accurate). Accuracy is only one evaluation measure used to score performance of your model and should be considered in conjunction with [precision](#precision) and [recall](#recall). | ||
| In [classification](#classification), accuracy is the number of correctly classified items divided by the total number of items in the test set. Ranges from 0 (least accurate) to 1 (most accurate). Accuracy is one of evaluation metrics of the performance of your model. Consider it in conjunction with [precision](#precision), [recall](#recall), and [F-score](#f-score). | ||
|
|
There was a problem hiding this comment.
This is fine for now, but we'll need to break this down into accuracy for binary classification and micro and macro accuracy for multi-classification later.
|
|
||
| A value that represents the area under the curve when false positives are plotted on the x-axis and true positives are plotted on the y-axis. Ranges from 0.5 (worst) to 1 (best). | ||
| In [binary classification](#binary-classification), an evaluation metric that is the value of the area under the curve that plots the true positives rate (on the y-axis) against the false positives rate (on the x-axis). Ranges from 0.5 (worst) to 1 (best). Also known as the area under the ROC curve, i.e., receiver operating characteristic curve. For more information, see the [Receiver operating characteristic](https://en.wikipedia.org/wiki/Receiver_operating_characteristic) article on Wikipedia. | ||
|
|
|
|
||
| When the data are being used to predict a category, [supervised learning](#supervised-learning) is also called classification. [Binary classification](#binary-classification) refers to predicting only two categories (for example assigning an image as a picture of either a 'cat' or a 'dog'). [Multiclass classification](#multiclass-classification) refers to predicting multiple categories (for example, when classifying an image as a specific breed of dog). | ||
| When the data is used to predict a category, [supervised learning](#supervised-learning) is also called classification. [Binary classification](#binary-classification) refers to predicting only two categories (for example, classifying an image as a picture of either a 'cat' or a 'dog'). [Multiclass classification](#multiclass-classification) refers to predicting multiple categories (for example, when classifying an image as a picture of a specific breed of dog). | ||
|
|
|
|
||
| A single number that indicates how well data fits a model. A value of 1 means that the model exactly matches the data. A value of 0 means that the data is random or otherwise cannot be fit to the model. This is often referred to as r<sup>2</sup>, R<sup>2</sup>, or r-squared. | ||
| In [regression](#regression), an evaluation metric that indicates how well data fits a model. Ranges from 0 to 1. A value of 0 means that the data is random or otherwise cannot be fit to the model. A value of 1 means that the model exactly matches the data. This is often referred to as r<sup>2</sup>, R<sup>2</sup>, or r-squared. | ||
|
|
|
|
||
| A measurable property of the phenomenon being measured, typically a numeric (double value). Multiple features are referred to as a **Feature vector** and typically stored as `double[]`. Features define the important characteristics about the phenomenon being measured. For more information see the [Feature](https://en.wikipedia.org/wiki/Feature_(machine_learning)) article on Wikipedia. | ||
| A measurable property of the phenomenon being measured, typically a numeric (double) value. Multiple features are referred to as a **Feature vector** and typically stored as `double[]`. Features define the important characteristics of the phenomenon being measured. For more information, see the [Feature](https://en.wikipedia.org/wiki/Feature_(machine_learning)) article on Wikipedia. | ||
|
|
|
|
||
| Feature engineering is the process of developing software that converts other data types (records, objects, …) into feature vectors. The resulting software performs Feature Extraction. For more information see the [Feature engineering](https://en.wikipedia.org/wiki/Feature_engineering) article on Wikipedia. | ||
| Feature engineering is the process that involves defining a set of [features](#feature) and developing software that produces feature vectors from available phenomenon data, i.e., feature extraction. For more information, see the [Feature engineering](https://en.wikipedia.org/wiki/Feature_engineering) article on Wikipedia. | ||
|
|
There was a problem hiding this comment.
This is definitely better than what we have now, but I think we'll need to write a little more on this eventually.
|
|
||
| An evaluation metric that balances [precision](#precision) and [recall](#recall). | ||
| In [classification](#classification), an evaluation metric that balances [precision](#precision) and [recall](#recall). | ||
|
|
|
|
||
| Parameters of machine learning algorithms. Examples include the number of trees to learn in a decision forest or the step size in a gradient descent algorithm. These parameters are called *Hyperparameters* because the process of learning is the process of identifying the right parameters of the prediction function. For example, the coefficients in a linear model or the comparison points in a tree. The process of finding those parameters is governed by the Hyperparameters. For more information see the [Hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter) article on Wikipedia. | ||
| A parameter of a machine learning algorithm. Examples include the number of trees to learn in a decision forest or the step size in a gradient descent algorithm. Values of *Hyperparameters* are set before training the model and govern the process of finding the [model](#model). For more information, see the [Hyperparameter](https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)) article on Wikipedia. | ||
|
|
There was a problem hiding this comment.
"finding the model" seems unclear here. I would leave in the examples that we mention in the definition and say that the hyper parameters help you find those parameters.
|
|
||
| Loss refers to an algorithm and task-specific measure of accuracy of the model on the training data. Log loss is the logarithm of the same quantity. | ||
| In [classification](#classification), an evaluation metric that characterizes the accuracy of a classifier. The smaller log loss is, the more accurate a classifier is. | ||
|
|
|
|
||
| An evaluation metric that averages all the model errors, where error is the predicted value distance from the true value. | ||
| In [regression](#regression), an evaluation metric that is the average of all the model errors, where model error is the distance between the predicted [label](#label) value and the correct label value. | ||
|
|
|
|
||
| Traditionally, the parameters for the prediction function. For example, the weights in a linear model or the split points in a tree. In ML.NET, a model contains all the information necessary to predict the label of a domain object (for example, image or text). This means that ML.NET models include the featurization steps necessary as well as the parameters for the prediction function. | ||
| Traditionally, the parameters for the prediction function. For example, the weights in a linear regression model or the split points in a decision tree. In ML.NET, a model contains all the information necessary to predict the [label](#label) of a domain object (for example, image or text). This means that ML.NET models include the featurization steps necessary as well as the parameters for the prediction function. | ||
|
|
|
|
||
| An evaluation metric that represents the error as a percentage of the true value. | ||
| In [regression](#regression), an evaluation metric that is the sum of all absolute errors divided by the sum of distances between correct [label](#label) values and the average of all correct label values. | ||
|
|
There was a problem hiding this comment.
I think for these evaluation metrics long term we should probably put in equations, as trying to parse through the wording (although accurate) can get confusing.
There was a problem hiding this comment.
@aditidugar that's true. I've also thought about inserting the link to related ML.NET API, so folks know immediately what to use from the library for an evaluation metric. And now, API remarks already contain equations (though not in pretty LaTeX format, but fine). If that's a good idea, I'll make a separate PR with such an update.
There was a problem hiding this comment.
Though, I haven't found API for the relative absolute error, but other evaluation metrics (RMS, RSquared, recall, etc) are supported by the library.
There was a problem hiding this comment.
@pkulikov yes, I think that's a good idea! Although, maybe I am not looking in the right place, but I don't seem to see a lot of the equations laid out in the API reference either (for example, https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.models.classificationmetrics.accuracymacro?view=ml-dotnet).
Either location (API ref or glossary itself) seems fine for the equation, but we should make sure all of them are there.
There was a problem hiding this comment.
OK, then I'll make another PR with API links.
As for the equations, we'll find later a good place for them.
|
@aditidugar thank you for the review! I've addressed your feedback on the Hyperparameter definition, please check the updated version. |
There was a problem hiding this comment.
LGTM. Thanks, @pkulikov, for this contribution, and thanks, @aditidugar for the thoughtful feedback! I'll get it merged straightaway, and it should show up on live sometime within the next few days.
5 participants
Made changes:
/cc: @aditidugar