Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hide adversarial parameters from model checkpoint. #150

Merged
merged 1 commit into from
Jun 1, 2023
Merged

Hide adversarial parameters from model checkpoint. #150

merged 1 commit into from
Jun 1, 2023

Conversation

mzweilin
Copy link
Contributor

@mzweilin mzweilin commented Jun 1, 2023 •
edited
Loading

What does this PR do?

Hides adversary parameters from model checkpoint.

Depends on #146

Type of change

Please check all relevant options.

  • Improvement (non-breaking)
  • Bug fix (non-breaking)
  • New feature (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

  • pytest
  • python -m mart experiment=CIFAR10_CNN_Adv trainer=gpu achieves 70% accuracy.

Before submitting

  • The title is self-explanatory and the description concisely explains the PR
  • My PR does only one thing, instead of bundling different changes together
  • I list all the breaking changes introduced by this pull request
  • I have commented my code
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have run pre-commit hooks with pre-commit run -a command without errors

Did you have fun?

Make sure you had fun coding 🙃

@mzweilin mzweilin requested a review from dxoigmn June 1, 2023 00:48
@mzweilin mzweilin mentioned this pull request Jun 1, 2023
Merged
14 tasks
dxoigmn
dxoigmn approved these changes Jun 1, 2023
View reviewed changes
Copy link
Contributor

@dxoigmn dxoigmn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@mzweilin mzweilin merged commit 48520cf into better_perturber Jun 1, 2023
@mzweilin mzweilin deleted the hide_adv_perturber_param branch June 1, 2023 15:36
dxoigmn added a commit that referenced this pull request Jun 1, 2023
@dxoigmn @mzweilin
Perturber as nn.Module (#146)
7551779 
* Make Perturber a standard nn.Module.
* Move GradientModifier from Perturber to Adversary.
* Hide Adversary parameters from model checkpoint. (#150)

---------

Co-authored-by: Weilin Xu <weilin.xu@intel.com>
mzweilin added a commit that referenced this pull request Jun 8, 2023
@mzweilin @dxoigmn
Make the Adversary Trainer accept loggers. (#164)
ed89c72 
* Remove NoAdversary

* Remove NoAdversary from CIFAR10 adversarial training

* Remove NoAdversary from RetinaNet model

* Fix COCO_TorchvisionFasterRCNN_Adv experiment

* Remove NoAdversary tests

* First stab at treating adversary as LightningModule

* style

* bugfix

* Integrate Perturber into LitPerturber

* Integrate objective to LitPerturber

* Cleanup use of objective function to compute gain

* bugfix

* Make adversarial trainer silent

* style

* Move threat model into LitPerturber

* Make attack callbacks plain PL callbacks

* comment

* Remove Perturber

* comment

* Better silence

* Integrate LitPerturber into Adversary

* Uncombine Adversary into Adversary and LitPerturber

* cleanup

* Move silence into utils

* bugfix

* bugfix

* cleanup

* Enable dependency injection on Adversary

* Make dependency injection backwards compatible

* Replace max_iters with trainer.limit_train_batches

* comments

* Move perturbation creation into initializer

* Add Default projector

* bugfix

* comment

* Move gradient modifier into PL hook

* Use on_train_epoch_start in favor of initialize_parameters

* Make perturbation lazy

* Disable logger in attack

* Revert initializer to d33658f

* cleanup

* on_before_optimizer_step -> configure_gradient_clipping

* comments

* Disable attack progress bar

* comments

* comments

* comments

* cleanup

* comments

* cleanup

* comments

* comment

* Move LitPerturber into perturber.py

* bugfix

* Make gradient modifiers in-place operations

* cleanup

* Mark initializers __call__ as no_grad instead of using .data

* Mark projectors __call__ as no_grad instead of using .data

* Cleanup attack configs

* Fix merge error

* Fix merge error

* comment

* Make Enforcer accept **kwargs.

* Update test_gradient.

* LitPerturber -> Perturber

* cleanup

* Add _reset functionality

* Update tests and fix a bug

* Remove batch tests

* style

* Late bind trainer to input device

* fix visualizer test

* bugfix

* bugfix

* disable progress bar

* bugfix

* Add loss to object detection outputs

* comment

* Make Adversary and Perturber tuple-aware

* comment

* Update tests and fix bug

* style

* Remove BatchEnforcer and BatchComposer

* Revert to old gain functionality

* Revert change to enforcer

* fix perturber tests to take gain

* cleanup

* Place Trainer on same device as Perturber

* Make composer, enforcer and projector tuple aware

* fix projector tests

* Gracefully fail when input is a dict

* Make Projector batch aware

* Update projector tests

* Fix Adversary gradient test

* Make attacker a property

* Fix configuration to construct Perturber

* Remove MaskAdditive

* Revert "Remove MaskAdditive"

This reverts commit d0ae325.

* cleanup

* Undelete

* Merge Perturber into Adversary

* Update projector test to use proper spec

* Abstract Perturber again

* bugfix

* Smarter configure_perturbation

* Make perturbations proper parameters and cleanup Initializer

* Cleanup GradientModifier

* Cleanup Composer

* Cleanup Projector

* Cleanup Enforcer

* Cleanup callbacks

* Cleanup NormalizedAdversaryAdapter

* Cleanup MartToArtAttackAdapter

* Smarter detection of when we need to create perturbation

* cleanup

* style

* Remove GradientModifier from Perturber and cleanup

* bugfix

* Add GradientModifier to LitModular

* Adversary consumes a OptimizerFactory

* Better Composer and Projector type logic

* spelling

* bugfix

* Replace tuple with Iterable[torch.Tensor]

* cleanup

* Fix tests

* Cleanup

* Make GradientModifier accept Iterable[torch.Tensor]

* Revert changes to LitModular

* Revert Adversary consumes a OptimizerFactory

* Remove Callback base

* Fix tests

* bugfix

* bugfix

* style

* Make attack callbacks normal callbacks

* Add gradient monitor callback

* bugfix

* Fix annotations

* Make Perturber more flexible

* bugfix

* Add GradientModifier and fix tests

* Fix configs

* Get adversary tests from adversary_as_lightningmodule

* Make attack callbacks normal callbacks

* Move attack optimizers to optimizers

* bugfix

* style

* comment

* fix test

* style

* style

* Perturber is no longer a callback

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* bugfix

* bugfix

* fix tests

* add missing tests

* return tests to original tests

* style

* Set optimizer to maximize in attacks

* Revert "Set optimizer to maximize in attacks"

This reverts commit 41eb387.

* Adversary optimizer maximizes gain

* Move Composer from Perturber and into Attacker

* cleanup

* bugfix

* Add Composer to Adversary

* cleanup

* projector -> projector_

* Hide adversarial parameters from model checkpoint. (#150)

* Fix merge error

* Pin numpy to 1.23.5 due to torch/tensorboard compatibility issue

* Make the Adversary Trainer accept loggers.

---------

Co-authored-by: Cory Cornelius <cory.cornelius@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dxoigmn dxoigmn dxoigmn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mzweilin @dxoigmn