Skip to content

v1.5

Latest
Latest
Compare
Choose a tag to compare
@mees mees released this 24 May 00:28
· 2 commits to main since this release
v1.5
5eaa5c6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

What's Changed

  • Improved cross-attention between visual and language tokens by repeating language tokens at every timestep in the context window.
  • Augmented the language instructions in the data with rephrasings from GPT-3.5.
  • Bug fixes:
    • Turned off dropout in the diffusion head due to incompatibility with layer norm.
    • Fixed an off-by-one error with the attention mask.
    • Fixed an issue where different image augmentations did not get fresh random seeds.
Assets 2
Loading