-
Notifications
You must be signed in to change notification settings - Fork 384
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FEAT Add image generation example with red teaming orchestrator and unify existing orchestrator definitions #189
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
romanlutz
added
documentation
Improvements or additions to documentation
enhancement
New feature or request
labels
May 4, 2024
6 tasks
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rdheekonda
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 6, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
reviewed
May 17, 2024
rlundeen2
approved these changes
May 17, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This is based on #82 by @ysy970923 and related to #74.
The goal of this PR was to create a notebook example for image generation where we use a model to generate prompts that we pass to the image generation endpoint. We then score the response and pass the scoring result back to the prompt generating model. If the score indicates that we succeeded we can stop. If a prompt gets blocked (due to content filters) we can pass that information back to the model as well so that it can generate a better prompt.
While making the adjustments from the original PR to support this natively within the red teaming orchestrator it became evident that the additional complexity we introduced to support
EndTokenRedTeamingOrchestrator
would make maintaining this going forward even tougher. Since that orchestrator can be supported using a scoring red teaming orchestrator (with small modifications), a large part of the changes in this PR are about unifying the definitions into a single RedTeamingOrchestrator that uses a scorer and now also supports images.[FYI @rlundeen2 as mentioned a week ago]
In this process, I discovered that some validation code (e.g., in the DALLE Target) may not be working as intended. The target should know what to send. So if it can only accept 1 prompt request piece then we should send just that rather than failing validation. The normalizer is currently unaware of what the target may or may not accept and can't adjust. FYI @rdheekonda
Apart from that, the DALLE Target needs to handle errors slightly differently. When no image was generated, the data type has to be text since we don't have an image_path for an image to store. Otherwise, the memory code raises exceptions. FYI @jbolor21
Tests and Documentation
Note that tests and other notebooks have yet to be adjusted.