Finding Custom Edit Directions

[lilu-REA]

hi ,thanks for your sharing.
about custom own sentence, can you also show the sentence examples and usually how do you create a large amount of sentence?

[sayakpaul]

We have a detailed example here. Does this help?

[lilu-REA]

thanks, it is clear

[rahulvigneswaran]

@sayakpaul @lilu-REA GPT-3.5 sentences seems to be generally better than FLAN. So I created directions for all 100 classes in CIFAR100. Take a look at this - https://github.com/rahulvigneswaran/pix2pix-zero-directions

[sayakpaul]

Cc: @gante

[gante]

Side note -- LLMs are evolving very fast, so it's natural that we can improve directions with the new models. For instance, Llama + Alpaca will likely be superior to FlanT5 :D

[rahulvigneswaran]

@gante True. My aim was to create a list of easy-to-access pre-generated directions for the 100 classes in cifar100. Do you know any code routine similar to @sayakpaul's but uses Llama+Alpaca to generate the same? I ran out of free GPT credits.

[gante]

@rahulvigneswaran

Alpaca + Llama in particular is tricky to share, due to licensing, but there are instructions in this repo.

[rahulvigneswaran]

@gante GPT sentences seem to be much more complex than Alpaca Lora - https://wandb.ai/rahulvigneswaran/Tailfusion/reports/Alpaca-LORA-Vs-GPT-3-5--VmlldzozOTEwODY4?accessToken=s3zeca4xsi03pb3mz539vwf0h1eagea35wqlb4po61e6gni6fu245im77uh9rdcj

Would using higher param count models of Alpaca Lora help with this?

[gante]

@rahulvigneswaran The repository I linked above has a very low temperature by default -- have you tried increasing it? Also, set num_beams to 1, beam search reduces the diversity of the output.

(Disclaimer -- I haven't played with the model myself, so maybe it sucks for this task! :D)

[rahulvigneswaran]

@gante Unfortunately, this doesn't seem to be helping :( . Would using a bigger model help? Is it worth the try?

[gante]

Definitely, a larger model should produce better results!

BTW, also give BLOOMZ a try :)

import torch
from transformers import AutoTokenizer, BloomForCausalLM

tokenizer = AutoTokenizer.from_pretrained("bigscience/bloomz-7b1")
model = BloomForCausalLM.from_pretrained("bigscience/bloomz-7b1", device_map="auto", torch_dtype=torch.float16)

input_text = "Provide a caption for images containing a cat. The captions should be in English and should be no longer than 150 characters. Caption:"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
input_length = input_ids.shape[1]

t = 0.8
eta = 1e-4
min_length = 10

print(f"\nSampling; temperature = {t}; eta_cutoff = {eta}, min_length={min_length}")
outputs = model.generate(
    input_ids,
    temperature=t,
    num_return_sequences=16,
    do_sample=True,
    max_new_tokens=128,
    min_length=min_length,
    eta_cutoff=eta
)
print("\n".join(tokenizer.batch_decode(outputs[:, input_length:], skip_special_tokens=True)))

[rahulvigneswaran]

@pix2pixzero Are sentences generated by Alpaca Lora like in https://wandb.ai/rahulvigneswaran/Tailfusion/reports/Alpaca-LORA-Vs-GPT-3-5--VmlldzozOTEwODY4?accessToken=s3zeca4xsi03pb3mz539vwf0h1eagea35wqlb4po61e6gni6fu245im77uh9rdcj good enough to generate edit directions?