Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FiftyOne callbacks #213

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Conversation

lumurillo
Copy link
Contributor

@lumurillo lumurillo commented Aug 2, 2023

What does this PR do?

Add two different FiftyOne callbacks:

  • FiftyOnePredictionAdder: Callback to store predictions to the corresponding FiftyOne dataset.
  • FiftyOneEvaluateDetections: Callback used to perform an evaluation of detection in some FiftyOne dataset.

The predictions and evaluation metrics can be automatically visualized into the FiftyOne app:
image

This PR is dependant of PR #209

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • pytest
  • CUDA_VISIBLE_DEVICES=0 python -m mart experiment=CIFAR10_CNN_Adv trainer=gpu trainer.precision=16 reports 70% (21 sec/epoch).
  • CUDA_VISIBLE_DEVICES=0,1 python -m mart experiment=CIFAR10_CNN_Adv trainer=ddp trainer.precision=16 trainer.devices=2 model.optimizer.lr=0.2 trainer.max_steps=2925 datamodule.ims_per_batch=256 datamodule.world_size=2 reports 70% (14 sec/epoch).

Before submitting

  • The title is self-explanatory and the description concisely explains the PR
  • My PR does only one thing, instead of bundling different changes together
  • I list all the breaking changes introduced by this pull request
  • I have commented my code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have run pre-commit hooks with pre-commit run -a command without errors

Known issues

When using the GPU device, the trainer.predict throws the following error:

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

The input data is on GPU, but the model's weights is on CPU.

@lumurillo lumurillo self-assigned this Aug 2, 2023
@lumurillo lumurillo force-pushed the lumurillo/add-fiftyone-datamodule branch from 6b718c8 to e31e0cb Compare August 24, 2023 20:28
Base automatically changed from lumurillo/add-fiftyone-datamodule to main August 24, 2023 20:40
@lumurillo lumurillo force-pushed the lumurillo/add-fiftyone-callbacks branch from 0d349c5 to 00bde06 Compare August 24, 2023 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant