Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat (ptq): activation equalization support #541

Merged
merged 6 commits into from
Jun 21, 2023
Merged

Feat (ptq): activation equalization support #541

merged 6 commits into from
Jun 21, 2023

Conversation

Giuseppe5
Copy link
Collaborator

@Giuseppe5 Giuseppe5 commented Feb 20, 2023 •
edited
Loading

Changes:

  • walk_region extended to keep track of activations encoutered in the path,
  • walk_region can walk through scale varying activations (e.g., sigmoid)

Activation Equalization support:

  • Possibility to set and tune alpha
  • For now, if there are multiple activations in one region, no equalization is performed (it could be mathematically possible but I have doubt about its usefulness)
  • If scale varying activations are used, then a standalone MUL op is added after the activation.

Missing tests. On a FP model, this transformation does not alter the math of the model, similar to graph equalization. Tests will be performed in a similar fashion.

@Giuseppe5 Giuseppe5 force-pushed the activation_eq_fp branch 2 times, most recently from 75bd26c to cb0ca25 Compare February 21, 2023 10:17
@Giuseppe5 Giuseppe5 force-pushed the activation_eq_fp branch 3 times, most recently from 229cb89 to 38f6e4d Compare March 17, 2023 17:21
@Giuseppe5 Giuseppe5 force-pushed the activation_eq_fp branch 3 times, most recently from a1297e8 to 5001efc Compare March 31, 2023 13:48
@Giuseppe5 Giuseppe5 force-pushed the activation_eq_fp branch 3 times, most recently from b261d9a to edb944c Compare April 11, 2023 11:20
@Giuseppe5 Giuseppe5 requested a review from volcacius June 21, 2023 08:39
@volcacius volcacius changed the title Feat: Activation Equalization support Feat (ptq): activation equalization support Jun 21, 2023
@volcacius volcacius merged commit fbe8670 into Xilinx:dev Jun 21, 2023
@Giuseppe5 Giuseppe5 deleted the activation_eq_fp branch February 15, 2024 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@volcacius volcacius volcacius approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Giuseppe5 @volcacius