Generating Slogans with Linguistic Features

LexPOS is a sequence-to-sequence transformer model that generates slogans with phonetic and structural repetition. For phonetic repetition, it searches for phonetically similar words with user keywords. Both the sound-alike words and user keywords become the lexical constraints while generating slogans. It also adjusts the logits distribution to implement further phonetic constraints. For structural repetition, LexPOS uses POS constraints. Users can specify any repeated phrase structure by POS tags.

1. Generating slogans with lexical, POS constraints

1.1. Code

# clone this repo
git clone https://github.com/yeounyi/LexPOS
cd LexPOS
# generate slogans 
python3 generate_slogans.py -keywords cake -num_beams 3 -temperature 1.2

• -keywords: Keywords that you want to be included in slogans. You can enter multiple keywords, delimited by comma
• -pos_inputs: You can either specify the particular list of POS tags delimited by comma, or the model will generate slogans with the most frequent syntax used in corpus. POS tags should follow the format of Universal POS tags.
• -num_beams: Number of beams for beam search. Default to 1, meaning no beam search.
• -temperature: The value used to module the next token probabilities. Default to 1.0.
• -model_path: Path to the pretrained model

1.2. Examples

Keyword: cake
POS: [VERB, DET, NOUN, PUNCT, VERB, DET, NOUN, PUNCT]
Output: Bake a cake, bake a smile

Keyword: computer
POS: [ADJ, NOUN, PUNCT, ADJ, NOUN, PUNCT]
Output: Superior Computer. Super peculiar.

Keywords: comfortable, furniture
POS: [NOUN, ADP, NOUN, PUNCT, NOUN, ADP, NOUN, PUNCT]
Output: Signature of comfortable furniture. Signature of style.

2. Generating slogans with lexical, POS, and phonetic constraints

2.1. Code

# clone this repo
git clone https://github.com/yeounyi/LexPOS
cd LexPOS/Phonetic_Constraints_During_Inference
# generate score matrix representing phonetic similarity of each token
python3 score_matrix.py
# generate slogans 
python3 generate_slogans_with_phonetic_constraints.py -keywords vacation,island -num_beams 3 -temperature 1.2 -alpha 10

• -keywords: Keywords that you want to be included in slogans. You can enter multiple keywords, delimited by comma
• -pos_inputs: You can either specify the particular list of POS tags delimited by comma, or the model will generate slogans with the most frequent syntax used in corpus. POS tags should follow the format of Universal POS tags.
• -num_beams: Number of beams for beam search. Default to 1, meaning no beam search.
• -temperature: The value used to module the next token probabilities. Default to 1.0.
• -model_path: Path to the pretrained model
• -alpha: Weights of phonetical constraints. Default to 0.0, meaning no phonetic constraints.

2.2. Examples

Keywords: vacation,island
POS: [VERB, DET, NOUN, PUNCT, VERB, DET, NOUN, PUNCT]
alpha 0: Make an occasion, vacation the island.
alpha 10: Take an island vacation, every occasion.

Keywords: water,pollution
POS: [NOUN, ADP, NOUN, PUNCT, NOUN, ADP, NOUN, PUNCT]
alpha 0: Pollution, without water, without depletion.
alpha 10: Pollution of water, depletion of water.

Model Architecture

Pretrained Models

1. LexPOSBart

• Trained with 2 lexical constraints
• Trained with lexical constraints with fixed order

2. LexPOSBart_multi

• Trained with 2-4 lexical constraints
• Trained with shuffled lexical constraints

References

https://github.com/aparrish/phonetic-similarity-vectors/

Citation

@misc{yi2021lexpos,
  author = {Yi, Yeoun},
  title = {Generating Slogans with Linguistic Constraints using Sequence-to-Sequence Transformer},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/yeounyi/LexPOS}}
}