Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unnecessary endpoint logic, rename collaborative to hivemind #13392

Merged
merged 7 commits into from
Jun 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source-pytorch/common_usecases.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ Customize and extend Lightning for things like custom hardware or distributed st
:header: Train on multiple machines over the internet
:description: Train on local machines or unreliable GPUs across the internet.
:col_css: col-md-12
:button_link: strategies/collaborative_training
:button_link: strategies/hivemind
:height: 100

.. displayitem::
Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/extensions/strategy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The below table lists all relevant strategies available in Lightning with their
- Strategy for training using the Bagua library, with advanced distributed training algorithms and system optimizations. :ref:`Learn more. <accelerators/gpu_intermediate:Bagua>`
* - collaborative
- :class:`~pytorch_lightning.strategies.HivemindStrategy`
- Strategy for training collaboratively on local machines or unreliable GPUs across the internet. :ref:`Learn more. <strategies/collaborative_training:Training on unreliable mixed GPUs across the internet>`
- Strategy for training collaboratively on local machines or unreliable GPUs across the internet. :ref:`Learn more. <strategies/hivemind:Training on unreliable mixed GPUs across the internet>`
* - fsdp
- :class:`~pytorch_lightning.strategies.DDPFullyShardedStrategy`
- Strategy for Fully Sharded Data Parallel provided by FairScale. :ref:`Learn more. <advanced/model_parallel:Fully Sharded Training>`
Expand Down
4 changes: 2 additions & 2 deletions docs/source-pytorch/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ Current Lightning Users
clouds/cluster
Save and load model progress <common/checkpointing>
Save memory with half-precision <common/precision>
Training over the internet <strategies/collaborative_training>
Training over the internet <strategies/hivemind>
advanced/model_parallel
clouds/cloud_training
Train on single or multiple GPUs <accelerators/gpu>
Expand Down Expand Up @@ -248,7 +248,7 @@ Current Lightning Users
Metrics <https://torchmetrics.readthedocs.io/en/stable/>
Model <model/build_model.rst>
Model Parallel <advanced/model_parallel>
Collaborative Training <strategies/collaborative_training>
Collaborative Training <strategies/hivemind>
Plugins <extensions/plugins>
Progress bar <common/progress_bar>
Production <deploy/production_advanced>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.. _collaborative_training:
.. _hivemind:

#####################################################
Training on unreliable mixed GPUs across the internet
Expand All @@ -17,23 +17,23 @@ Training on unreliable mixed GPUs across the internet
:header: 1: Training across multiple machines over the internet
:description: Quick setup to start training on multiple machines.
:col_css: col-md-4
:button_link: collaborative_training_basic.html
:button_link: hivemind_basic.html
:height: 200
:tag: basic

.. displayitem::
:header: 2: Speed up training by enabling under-the-hood optimizations
:description: Learn which flags to use with the HivemindStrategy to speed up training.
:col_css: col-md-4
:button_link: collaborative_training_intermediate.html
:button_link: hivemind_intermediate.html
:height: 200
:tag: intermediate

.. displayitem::
:header: 3: Optimize Memory and Communication using compression hooks
:description: Enable gradient buffer optimizations and communication improvements to reduce bottlenecks in communication.
:col_css: col-md-4
:button_link: collaborative_training_expert.html
:button_link: hivemind_expert.html
:height: 200
:tag: expert

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
:orphan:

.. _collaborative_training_basic:
.. _hivemind_basic:

Training on unreliable mixed GPUs across the internet (Basic)
=============================================================
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
:orphan:

.. _collaborative_training_expert:
.. _hivemind_expert:

Training on unreliable mixed GPUs across the internet (Expert)
==============================================================
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
:orphan:

.. _collaborative_training_intermediate:
.. _hivemind_intermediate:

Training on unreliable mixed GPUs across the internet (Intermediate)
====================================================================
Expand Down
1 change: 1 addition & 0 deletions src/pytorch_lightning/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Hivemind Strategy
* Added `CollaborativeStrategy` ([#12842](https://github.com/PyTorchLightning/pytorch-lightning/pull/12842))
* Renamed `CollaborativeStrategy` to `HivemindStrategy` ([#13388](https://github.com/PyTorchLightning/pytorch-lightning/pull/13388))
* Removed unnecessary endpoint logic, renamed `collaborative` to `hivemind` ([#13392](https://github.com/PyTorchLightning/pytorch-lightning/pull/13392))

- Include a version suffix for new "last" checkpoints of later runs in the same directory ([#12902](https://github.com/PyTorchLightning/pytorch-lightning/pull/12902))

Expand Down
2 changes: 1 addition & 1 deletion src/pytorch_lightning/strategies/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from pytorch_lightning.strategies.bagua import BaguaStrategy # noqa: F401
from pytorch_lightning.strategies.collaborative import HivemindStrategy # noqa: F401
from pytorch_lightning.strategies.ddp import DDPStrategy # noqa: F401
from pytorch_lightning.strategies.ddp2 import DDP2Strategy # noqa: F401
from pytorch_lightning.strategies.ddp_spawn import DDPSpawnStrategy # noqa: F401
from pytorch_lightning.strategies.deepspeed import DeepSpeedStrategy # noqa: F401
from pytorch_lightning.strategies.dp import DataParallelStrategy # noqa: F401
from pytorch_lightning.strategies.fully_sharded import DDPFullyShardedStrategy # noqa: F401
from pytorch_lightning.strategies.fully_sharded_native import DDPFullyShardedNativeStrategy # noqa: F401
from pytorch_lightning.strategies.hivemind import HivemindStrategy # noqa: F401
from pytorch_lightning.strategies.horovod import HorovodStrategy # noqa: F401
from pytorch_lightning.strategies.hpu_parallel import HPUParallelStrategy # noqa: F401
from pytorch_lightning.strategies.ipu import IPUStrategy # noqa: F401
Expand Down
Loading