Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metric docs fix #2209

Merged
merged 34 commits into from
Jun 17, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
2d5b8e9
fix docs
Jun 16, 2020
5400081
Update docs/source/metrics.rst
williamFalcon Jun 16, 2020
de5083f
Update docs/source/metrics.rst
williamFalcon Jun 16, 2020
a902b30
Update docs/source/metrics.rst
williamFalcon Jun 16, 2020
c2145fa
Update docs/source/metrics.rst
williamFalcon Jun 16, 2020
c8d3a91
Update metrics.rst
williamFalcon Jun 16, 2020
4556f6e
title
Borda Jun 16, 2020
254b023
fix
Jun 16, 2020
6b11067
fix for num_classes
Jun 16, 2020
4cadc58
chlog
Borda Jun 16, 2020
63690bf
nb classes
Borda Jun 16, 2020
0ecc521
hints
Borda Jun 16, 2020
9f1cbd1
zero division
Borda Jun 16, 2020
46f59d9
add tests
Borda Jun 16, 2020
1660e42
Update metrics.rst
edenlightning Jun 16, 2020
2a407a9
Update classification.py
edenlightning Jun 16, 2020
81095a7
Update classification.py
edenlightning Jun 16, 2020
764b52a
prune doctests
Borda Jun 16, 2020
d5e1b31
Merge branch 'metric_docs' of https://github.com/SkafteNicki/pytorch-…
Borda Jun 16, 2020
770787e
docs
Borda Jun 16, 2020
8cd4a52
Apply suggestions from code review
Borda Jun 16, 2020
4de28c3
Apply suggestions from code review
Borda Jun 16, 2020
70f9162
flake8
Borda Jun 16, 2020
f6144ee
doctests
Borda Jun 16, 2020
7f201ee
formatting
Borda Jun 16, 2020
7701bd4
cleaning
Borda Jun 16, 2020
fe6699a
formatting
Borda Jun 16, 2020
1810dfe
formatting
Borda Jun 16, 2020
59e49a4
doctests
Borda Jun 16, 2020
f3a0fb3
flake8
Borda Jun 16, 2020
caaacf9
docs
Borda Jun 16, 2020
af872fc
rename
Borda Jun 17, 2020
f9ee46b
rename
Borda Jun 17, 2020
ae0b3c6
typo
Borda Jun 17, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions docs/source/metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ In this package we provide two major pieces of functionality.

1. A Metric class you can use to implement metrics with built-in distributed (ddp) support which are device agnostic.
2. A collection of popular metrics already implemented for you.
3. A interface to call `sklearns metrics <https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics>`_

Example::

Expand All @@ -28,6 +29,10 @@ Out::

tensor(0.7500)

.. warning:: The metrics package is still in development, and is the moment limited
williamFalcon marked this conversation as resolved.
Show resolved Hide resolved
to a few metrics. Please feel free to create an issue/PR if you have a proposed
metric or have found a bug.

--------------

Implement a metric
Expand Down Expand Up @@ -316,3 +321,98 @@ to_onehot (F)

.. autofunction:: pytorch_lightning.metrics.functional.to_onehot
:noindex:

----------------

Sklearn interface
-----------------

Lightning supports `sklearns metrics module <https://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics>`_
as a backend for calculating metrics. Sklearns metrics are well tested and robust,
but requires conversion between pytorch and numpy thus may slow down your computations.

To use the sklearn backend of metrics simply import as

Example::
williamFalcon marked this conversation as resolved.
Show resolved Hide resolved

import pytorch_lightning.metrics.sklearn import plm
metric = plm.Accuracy(normalize=True)
val = metric(pred, target)

Each converted sklearn metric comes has the precise same call interface as its
williamFalcon marked this conversation as resolved.
Show resolved Hide resolved
originally counterpart (e.g. accuracy takes the additional `normalize` keyword).
Like the native implemented metrics these converted sklearn metrics also comes
with build in distributed (ddp) support.
williamFalcon marked this conversation as resolved.
Show resolved Hide resolved

SklearnMetric (S)
SkafteNicki marked this conversation as resolved.
Show resolved Hide resolved
^^^^^^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

Accuracy (S)
^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

AUC (S)
^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

AveragePrecision (S)
^^^^^^^^^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:


ConfusionMatrix (S)
^^^^^^^^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

F1 (S)
^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

FBeta (S)
^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

Precision (S)
^^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

Recall (S)
^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

PrecisionRecallCurve (S)
^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

ROC (S)
^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:

AUROC (S)
^^^^^^^^^

.. autofunction:: pytorch_lightning.metrics.sklearn.SklearnMetric
:noindex:
2 changes: 1 addition & 1 deletion pytorch_lightning/metrics/classification.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ def __init__(
>>> target = torch.tensor([0, 1, 2, 2])
>>> metric = Precision()
>>> metric(pred, target)
tensor(1.)
tensor(0.75)
Borda marked this conversation as resolved.
Show resolved Hide resolved

"""
super().__init__(name='precision',
Expand Down